Bug 3425 – StdioException on end of stdin on Windows

Code: import std.stdio, std.algorithm; void main(string[] args) { auto foo = stdin.byLine(); foreach(elem; foo) { writeln(elem); } } Input file (foo.txt) : foo bar Result: F:\>cat foo.txt | test2.exe foo bar std.stdio.StdioException: Bad file descriptor Works perfectly on Linux. Haven't tested on Mac, BSD or whatever the heck else DMD runs on lately.

I've managed to figure out where this exception is coming from, but I don't know how to fix it because I don't know why errno is getting set. It's on line 2271 in stdio.d: if (fp._flag & _IONBF) { /* Use this for unbuffered I/O, when running * across buffer boundaries, or for any but the common * cases. */ L1: auto app = appender(buf); app.clear(); if(app.capacity == 0) app.reserve(128); // get at least 128 bytes available int c; while((c = FGETC(fp)) != -1) { app.put(cast(char) c); if(c == terminator) { buf = app.data; return buf.length; } } if (ferror(fps)) StdioException(); Can we **PLEASE** fix this one ASAP? It's been in Bugzilla for over a year and a half and makes it impossible to write simple text filter programs on Windows without ugly and/or unsafe workarounds.

You can work around the issue by testing for eof on the first line in the loop. This works with no error. foreach(elem; stdin.byLine()) { if (stdin.eof()) break; writeln(elem); }

(In reply to comment #2) > You can work around the issue by testing for eof on the first line in the > loop. This works with no error. > > foreach(elem; stdin.byLine()) { > if (stdin.eof()) break; > writeln(elem); > } This didn't work for me, when using the example program on the D homepage: import std.stdio; void main() { ulong lines = 0; double sumLength = 0; foreach(line; stdin.byLine()) { if (stdin.eof()) break; ++lines; sumLength += line.length; } writeln("Average line length = ", lines ? sumLength / lines : 0); } I still get: std.stdio.StdioException@std\stdio.d(2159): Bad file descriptor

However, I did find that if I modify stdio.d in Phobos to the following under the L1 label around line 2280, then the other form of piping, i.e. "type filename | executable", will work on Windows. if (ferror(fps) && EPIPE != ferror(fps)) StdioException(); In the "|" pipe case, it could be that perhaps the "type" command already closed down its end by the time ferror() was called on the receiving side. On Windows, "executable < filename" piping worked fine, but is not ideal for stringing filters together. Here's the writeln-debugged block of stdio.d code that works: // Private implementation of readln version (DIGITAL_MARS_STDIO) private size_t readlnImpl(FILE* fps, ref char[] buf, dchar terminator = '\n') { FLOCK(fps); scope(exit) FUNLOCK(fps); writeln("Entered readlnImpl()"); scope(exit) writeln("Exited readlnImpl()"); /* Since fps is now locked, we can create an "unshared" version * of fp. */ auto fp = cast(_iobuf*)fps; if (__fhnd_info[fp._file] & FHND_WCHAR) { /* Stream is in wide characters. * Read them and convert to chars. */ writeln("Entered if (__fhnd_info[fp._file] & FHND_WCHAR)"); static assert(wchar_t.sizeof == 2); auto app = appender(buf); app.clear(); for (int c = void; (c = FGETWC(fp)) != -1; ) { if ((c & ~0x7F) == 0) { app.put(cast(char) c); if (c == terminator) break; } else { if (c >= 0xD800 && c <= 0xDBFF) { int c2 = void; if ((c2 = FGETWC(fp)) != -1 || c2 < 0xDC00 && c2 > 0xDFFF) { StdioException("unpaired UTF-16 surrogate"); } c = ((c - 0xD7C0) << 10) + (c2 - 0xDC00); } //std.utf.encode(buf, c); app.put(cast(dchar)c); } } if (ferror(fps)) StdioException(); buf = app.data; return buf.length; } auto sz = GC.sizeOf(buf.ptr); //auto sz = buf.length; buf = buf.ptr[0 .. sz]; if (fp._flag & _IONBF) { writeln("Entered if (fp._flag & _IONBF)"); /* Use this for unbuffered I/O, when running * across buffer boundaries, or for any but the common * cases. */ L1: auto app = appender(buf); app.clear(); if(app.capacity == 0) app.reserve(128); // get at least 128 bytes available int c; writeln("fp._cnt: ", fp._cnt); while((c = FGETC(fp)) != -1) { writeln("chars: ", cast(char) c); app.put(cast(char) c); if(c == terminator) { writeln("hit terminator"); buf = app.data; return buf.length; } } writeln("feof(fps): ", feof(fps)); writeln("ferror(fps): ", ferror(fps)); // If EPIPE is seen then probably the other side has closed // already. This is the case when using, for example: // // "type filename | D-program" syntax on Windows. if (ferror(fps) && EPIPE != ferror(fps)) StdioException(); buf = app.data; return buf.length; } else { writeln("Entered if (!(fp._flag & _IONBF))"); int u = fp._cnt; char* p = fp._ptr; int i; writeln("length of stdin fp: ", u); if (fp._flag & _IOTRAN) { /* Translated mode ignores \r and treats ^Z as end-of-file */ writeln("Entered if (fp._flag & _IOTRAN)"); char c; while (1) { if (i == u) // if end of buffer goto L1; // give up c = p[i]; i++; if (c != '\r') { if (c == terminator) break; if (c != 0x1A) continue; goto L1; } else { if (i != u && p[i] == terminator) break; goto L1; } } if (i > sz) { buf = uninitializedArray!(char[])(i); } if (i - 1) memcpy(buf.ptr, p, i - 1); buf[i - 1] = cast(char)terminator; buf = buf[0 .. i]; if (terminator == '\n' && c == '\r') i++; } else { writeln("Entered if !(fp._flag & _IOTRAN)"); while (1) { if (i == u) // if end of buffer goto L1; // give up auto c = p[i]; i++; if (c == terminator) break; } if (i > sz) { buf = uninitializedArray!(char[])(i); } memcpy(buf.ptr, p, i); buf = buf[0 .. i]; } fp._cnt -= i; fp._ptr += i; return i; } } And here's what it outputs: c:\>avg < index.html Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 0 Entered if !(fp._flag & _IOTRAN) fp._cnt: 0 chars: < chars: h chars: t chars: m chars: l chars: > chars: hit terminator Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 106 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 97 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 37 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 27 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 18 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 8 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 0 Entered if !(fp._flag & _IOTRAN) fp._cnt: 0 feof(fps): 16 ferror(fps): 0 Exited readlnImpl() Average line length = 15.1429 c:\>type index.html | avg Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 0 Entered if !(fp._flag & _IOTRAN) fp._cnt: 0 chars: < chars: h chars: t chars: m chars: l chars: > chars: hit terminator Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 106 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 97 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 37 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 27 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 18 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 8 Entered if !(fp._flag & _IOTRAN) Exited readlnImpl() Entered readlnImpl() Entered if (!(fp._flag & _IONBF)) length of stdin fp: 0 Entered if !(fp._flag & _IOTRAN) fp._cnt: 0 feof(fps): 0 ferror(fps): 32 Exited readlnImpl() Average line length = 15.1429

It seems this problem doesn't exist anymore.

According to a user on the forums, this bug still occurs with DMD 2.067.1: http://forum.dlang.org/thread/[email protected]

works well on: Microsoft Windows [Version 10.0.14393] with: C:\Users\notna>dmd --version DMD32 D Compiler v2.073.1 Copyright (c) 1999-2016 by Digital Mars written by Walter Bright