Bug 2842 – std.file.listdir on OSX produces invalid UTF-8 sequence

std.file.listdir throws an "invalid UTF-8 sequence" exception. please see the source file and the produced output below. thanks. ------------------------------ import std.stdio, std.file; int main(char[][] args) { foreach (file; listdir(".")) writefln("%s", file); return 0; } ------------------------------ blah:misc user$ dmd listdir.d blah:misc user$ ./listdir .. some_directory Error: 1invalid UTF-8 sequence blah:misc user$

What does the "ls -al" show for that directory? Is it happening on all files? It works ok on v2.031 when run from the samples dir.

I can confirm this bug using DMD 1.050 on OS X 10.5. Running program by OP: emsi-testings-mac-mini:ratchet emsitesting$ ./listdir . . ogout . e istory . s . _central.d t re sion . . ac.sh inux.sh _central.o _central . .d .o . The function seems to be trimming the names arbitrarily. Here's ls -al in same directory: emsi-testings-mac-mini:ratchet emsitesting$ ls -al total 1324 drwx------ 0 emsitesting staff 16384 Nov 9 13:10 . drwxrwxrwt@ 5 root admin 170 Nov 5 11:35 .. -rw--w---- 1 emsitesting staff 6148 Nov 5 11:27 .DS_Store -rw------- 1 emsitesting staff 7467 Nov 9 12:36 .bash_history -rw-r--r-- 1 emsitesting staff 220 Nov 4 11:11 .bash_logout -rw-r--r-- 1 emsitesting staff 3211 Nov 4 13:32 .bashrc drwxr-xr-x 2 emsitesting staff 16384 Nov 4 13:27 .cache -rw-r--r-- 1 emsitesting staff 675 Nov 4 11:11 .profile -rw-r--r-- 1 emsitesting staff 81 Nov 6 12:20 .ratchet drwx------ 2 emsitesting staff 16384 Nov 4 13:53 .ssh drwxr-xr-x 3 emsitesting staff 16384 Nov 6 10:50 .subversion -rwxrwxr-x 1 emsitesting staff 228 Nov 6 13:41 build_linux.sh -rwxr-xr-x 1 emsitesting staff 187 Nov 6 14:17 build_mac.sh lrwxrwxrwx 1 emsitesting staff 6 Nov 6 10:50 ccb -> d/ccb/ lrwxrwxrwx 1 emsitesting staff 16 Nov 6 10:54 chains -> d/chains/chains/ drwxr-xr-x 26 emsitesting staff 16384 Nov 9 12:02 d -rwxr-xr-x 1 emsitesting staff 261192 Nov 9 13:10 listdir -rw-rw-r-- 1 emsitesting staff 138 Nov 9 13:10 listdir.d -rw-rw-r-- 1 emsitesting staff 2748 Nov 9 13:10 listdir.o drwxr-xr-x 6 emsitesting staff 16384 Nov 9 12:34 projects -rwxr-xr-x 1 emsitesting staff 260223 Nov 6 13:37 ratchet_central -rw-r--r-- 1 emsitesting staff 2331 Nov 6 13:37 ratchet_central.d -rw-r--r-- 1 emsitesting staff 9456 Nov 6 13:37 ratchet_central.o This one's rather critical as I have code that I'm trying to get running on OS X. (In reply to comment #1) > What does the "ls -al" show for that directory? > > Is it happening on all files? > > It works ok on v2.031 when run from the samples dir.

This bug still exists in 1.063.

*** Issue 6011 has been marked as a duplicate of this issue. ***

Still exists in 1.064, 1.065 and 1.068.

Looks like the cause of this is due to std.c.linux.linux is being used in std.file where dirent is defined to this: struct dirent { uint d_ino; // this is int on some linuxes off_t d_off; ushort d_reclen; ubyte d_type; // this field isn't there on some linuxes char[256] d_name; } While in /usr/include/sys/dirent.h it is defined to this: struct dirent { ino_t d_ino; /* file number of entry */ __uint16_t d_reclen; /* length of this record */ __uint8_t d_type; /* file type, see below */ __uint8_t d_namlen; /* length of string in d_name */ char d_name[__DARWIN_MAXNAMLEN + 1]; /* name must be no longer than this */ }; Cheers Jakob.

Created attachment 976 Patch that should fix the problem, untested.

Created attachment 977 Test program from other bug. Here is the test program from issue 6011. Cheers Jakob.