Bug 23312 – Crash when calling writeln in WinMain

Status
RESOLVED
Resolution
FIXED
Severity
normal
Priority
P3
Component
druntime
Product
D
Version
D2
Platform
x86_64
OS
Windows
Creation time
2022-08-29T14:50:05Z
Last change time
2023-06-21T18:25:00Z
Keywords
pull
Assigned to
No Owner
Creator
Perry
See also
https://issues.dlang.org/show_bug.cgi?id=24001

Comments

Comment #0 by alex — 2022-08-29T14:50:05Z
New to D, so maybe I'm doing something stupid here but I can't use writeln when I'm using win32. Here's a minimal example: ``` import core.runtime; import core.sys.windows.windows; import core.sys.windows.winuser; import std.string; import std.utf; import std.stdio; pragma(lib, "user32"); extern (Windows) int WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow) { int result; try { Runtime.initialize(); result = myWinMain(hInstance, hPrevInstance, lpCmdLine, nCmdShow); Runtime.terminate(); } catch (Throwable e) { MessageBox(null, e.toString().toUTF16z(), null, MB_ICONEXCLAMATION); result = 0; // failed } return result; } int myWinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow) { writeln("hello\n"); return 0; } ``` It causes a crash in the second `__setmode` of this part in `LockingTextWriter`: ``` version (MICROSOFT_STDIO) { // Microsoft doesn't implement fwide. Instead, there's the // concept of ANSI/UNICODE mode. fputc doesn't work in UNICODE // mode; fputwc has to be used. So that essentially means // "wide-oriented" for us. immutable int mode = __setmode(f.fileno, _O_TEXT); // Set some arbitrary mode to obtain the previous one. __setmode(f.fileno, mode); // Restore previous mode. if (mode & (_O_WTEXT | _O_U16TEXT | _O_U8TEXT)) { orientation_ = 1; // wide } } ``` I can use `OutputDebugString` instead but I should be able to use writeln safely right? Thanks
Comment #1 by andrej.mitrovich — 2022-09-25T11:13:14Z
My workaround for this is to use `main()` instead of `WinMain()`. It's still possible to write GUI apps this way, and there will be a console spawned with the app so the stdout/stderr handles are valid this way. Maybe there's a way to spawn a console when using WinMain if one is not detected. Or alternatively it would be nice to be able to set stdout/stderr to a file handle when using WinMain. Not sure if this is already possible.
Comment #2 by aldacron — 2022-09-25T12:20:07Z
This is not specific to D. It's a Windows thing, and it happens in C and C++ programs as well. The presence of WinMain in your source automatically tells most (all?) linkers on Windows that you're building a "windows subsystem" executable. This means standard I/O will be unavailable at startup since there's no console to write to or read from. Andrej's suggestion of using main instead, which is automatically interpreted as a "console subsystem" works, and is probably the best solution if you need stdout/stderr/stdin. Windows linkers also support a flag that allows you to specify either the windows or console subsystem (among others) no matter which entry point you have (WinMain or main). The MS linker also has a flag, and requires its use, to specify the entry point if it doesn't match the subsystem. You can find the documentation for both at the following links (and remember, these are specifically for the Microsoft linker; the LDC linker may support both or may not require the entry specification, I don't know): https://learn.microsoft.com/en-us/cpp/build/reference/subsystem-specify-subsystem?view=msvc-170 https://learn.microsoft.com/en-us/cpp/build/reference/entry-entry-point-symbol?view=msvc-170 But there's no reason to use WinMain if you want a console, so I just use main. Anyway, this isn't a D-specific issue, so I'm going to close this as invalid. If there's not a way in the standard library to redirect standard I/O (to file, or a console created via AttachConsole), there should be, but that should be a separate issue from this one.
Comment #3 by alex — 2022-09-26T09:20:45Z
Thanks I'll use main as a workaround. I'd argue this is still a bug, pretty sure in C/C++ the output would silently fail rather than outright crash.
Comment #4 by aldacron — 2022-09-26T10:01:56Z
Okay. I glossed over the bit about the crash. I'll change the title and reopen.
Comment #5 by alphaglosined — 2023-04-12T08:28:09Z
It appears on Microsoft's end that _setmode is expected to assert out if anything isn't right. For example: > Unicode mode is for wide print functions (for example, wprintf) and is not supported for narrow print functions. Use of a narrow print function on a Unicode mode stream triggers an assert. https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setmode?view=msvc-170 Realistically though std.stdio shouldn't be used for console output and therefore that particular bit of code shouldn't exist to assert out.
Comment #6 by bugzilla — 2023-06-21T08:32:47Z
The code that is crashing is calling _setmode() but not checking the error return from it. https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setmode?view=msvc-170
Comment #7 by dlang-bot — 2023-06-21T08:35:55Z
@WalterBright created dlang/phobos pull request #8770 "fix Issue 23312 - Crash when calling writeln in WinMain" fixing this issue: - fix Issue 23312 - Crash when calling writeln in WinMain https://github.com/dlang/phobos/pull/8770
Comment #8 by dlang-bot — 2023-06-21T18:25:00Z
dlang/phobos pull request #8770 "fix Issue 23312 - Crash when calling writeln in WinMain" was merged into master: - cdf9ce5e516199ced594ca00d9b6b7575193ee9b by Walter Bright: fix Issue 23312 - Crash when calling writeln in WinMain https://github.com/dlang/phobos/pull/8770