Bug 9776 – Make raw write mode the default

On Windows, the "write*" functions automatically convert \n to \r\n. This is both error-prone and completely unnecessary on all versions of windows that DMD/Phobos supports (ie, at *least* as far back as XP). On all such versions of Windows, the command line, BATch files and code editors all handle \n perfectly fine. If there are any obscure situations where \r\n is still expected, they are more properly handled as-needed as special cases. The big-prone nature of non-raw write is demonstrated in this straightforward code: -------------------------------------- import std.file; import std.stdio; void transform(string str) { /+ ...perform some modification of 'str'... +/ return str; } void main() { auto str = cast(string) read(args[1]); str = transform(str); write(str); } -------------------------------------- That simple code is *wrong*: It works correctly for all input on Unix: Output newlines match input newlines. Always. The code never asks for newlines to be messed with, and therefore they never are. But on Windows the behavior is just plain weird: Unix-style newlines in the input are silently and forcefully converted to Windows-style newlines behind the user's back. And even worse yet, *Windows*-style newlines on the input are converted to a bizarre "Mac9 plus Windows-style" combination of "\r\r\n" (not only is that wrong period, but this sequence is often interpreted as two newlines which makes it even worse). Using rawWrite fixes the problem, and creates *no* problem. People have been caught by that bug on various occasions, such as: https://github.com/repeatedly/mustache-d/issues/3 See also the discussion on this matter on the D newsgroup: http://forum.dlang.org/post/20130320103418.00005416@unknown Various people have agreed, and so far, there hasn't been anyone arguing to keep the current behavior.

*** Issue 11087 has been marked as a duplicate of this issue. ***

Simple test-case from Issue 11087: ----- import std.file; import std.stdio; void main() { std.file.write("test1.txt", "a\nb"); auto file2 = File("test2.txt", "w"); file2.write("a\nb"); file2.close(); auto res1 = cast(byte[])std.file.read("test1.txt"); auto res2 = cast(byte[])std.file.read("test2.txt"); writeln(res1); // writes [97, 10, 98] writeln(res2); // writes [97, 13, 10, 98] } -----

Another way of putting it. Do this on a Windows machine: --------------------------------------- > type test.d import std.file, std.stdio; void main() { auto str = cast(string) std.file.read("input.txt"); stdout.write(str); } > echo line1 > input.txt > echo line2 >> input.txt > dmd -run test.d > output.txt --------------------------------------- Then notice that "input.txt" and "output.txt" are DIFFERENT (Check in a hex viewer or any decent editor that supports visible line endings.) The input has \r\n, but the output has \r\r\n. Some editors will actually display it like this: --------------- line1 line2 ---------------

I seriously doubt this is going to be changed as long as we use the C I/O API. Not to mention that this would be a breaking change. (In reply to Nick Sabalausky from comment #0) > completely unnecessary on all versions of windows that > DMD/Phobos supports (ie, at *least* as far back as XP). Wrong, the Windows console still treats \n as "line feed without carriage return". (In reply to Andrej Mitrovic from comment #2) > auto file2 = File("test2.txt", "w"); Use "wb" as the file mode, this is what it's for. (In reply to Nick Sabalausky from comment #3) > auto str = cast(string) std.file.read("input.txt"); > stdout.write(str); Do not mix C I/O with OS I/O APIs.

(In reply to Vladimir Panteleev from comment #4) > Wrong, the Windows console still treats \n as "line feed without carriage > return". I just tested and it seems I misremembered, sorry. But it's still a breaking change.

(In reply to Vladimir Panteleev from comment #4) > (In reply to Nick Sabalausky from comment #3) > > auto str = cast(string) std.file.read("input.txt"); > > stdout.write(str); > > Do not mix C I/O with OS I/O APIs. It's not at all clear that std.file and std.stdio use completely different APIs internally. To the avrage D user, it's all just Phobos API. (In reply to Vladimir Panteleev from comment #5) > > But it's still a breaking change. So are most bugfixes.

I'm telling you, this is not going to be fixed without introducing a new API. Edit-warring the status is not going to help.

Whether you agree or disagree, everything here works "as designed", i.e. "do what C does". As explained, changing the default behaviour of File.open would be a breaking change and is not going to happen.

> unnecessary on all versions of windows Windows Notepad fails with it. It's similar to the \ path separator problem on Windows. Many programs, including lots of Microsoft programs, fail to deal with / properly.

As already said before, windows notepad is the *only program that doesn't handle it (which makes it very *unlike* '/' if indeed "many programs" fail to deal with properly - which is an exaggeration anyway, but that's beside the point). If I'm wrong and \n is indeed poorly supported on Windows, then it should be a simple matter to go ahead and name other examples. Contrast that to the "\r\r\n" that D code is demonstrably prone to emitting on Windows, which is handled correctly by little, if anything, on any OS. The safer behavior is clear, and the consensus of the discussion was the same. Stubborn clingage to false information and misplaced fears is not befitting of D.

*** Issue 4555 has been marked as a duplicate of this issue. ***

(In reply to Vladimir Panteleev from comment #8) > Whether you agree or disagree, everything here works "as designed" Not documented though.

(In reply to anonymous4 from comment #12) > Not documented though. How so? The documentation for stdioOpenmode says "range or string represting the open mode (with the same semantics as in the C standard library fopen function)", so the defaults for unspecified options are left to the C standard library implementation. AIUI, it's not part of the language because on POSIX binary and text modes behave the same.

Not documented that std.file doesn't use C stdio and is not compatible with it (and not intended) and no remark to not mix it with stdio as was suggested here.