Bug 20012 – extern(C) functions inside template mixins are not mangled as C functions

This works: export extern(C) void fun() {} Build as a DLL, fun is correctly exported. You can get "fun" with GetProcAddress. This doesn't work: mixin template M() { export extern(C) void fun() {} } mixin M!(); When I do this, "fun" is just not exported and GetProcAddress fails. What's very strange is, the library feature SimpleDllMain in core.sys.windows.dll to mixin the DllMain function DOES seem to work, and looking at the source, it's not even declared `export`... so maybe there's just some special handling for the DllMain symbol?

It is exported, just as: __D4test8__mixin43fooUZv instead of: _foo You can see what's exported by running obj2asm on the .obj file, don't need to use GetProcAddress. For example: --- export extern(C) void fun() {} mixin template M() { export extern(C) void bar() {} } mixin M!(); --- dmd -c test.d obj2asm test.obj --- _TEXT segment dword use32 public 'CODE' ;size is 0 _TEXT ends _DATA segment para use32 public 'DATA' ;size is 0 _DATA ends CONST segment para use32 public 'CONST' ;size is 13 CONST ends _BSS segment para use32 public 'BSS' ;size is 0 _BSS ends FLAT group extrn _fun ;expdef expflag=x00, export '_fun', internal '', ordinal=x0 extrn __D4test8__mixin43barUZv ;expdef expflag=x00, export '__D4test8__mixin43barUZv', internal '', ordinal=x0 FMB segment dword use32 public 'DATA' ;size is 0 FMB ends FM segment dword use32 public 'DATA' ;size is 4 FM ends FME segment dword use32 public 'DATA' ;size is 0 FME ends public __D4test12__ModuleInfoZ _fun COMDAT flags=x0 attr=x0 align=x0 __D4test8__mixin43barUZv COMDAT flags=x0 attr=x10 align=x0 _TEXT segment assume CS:_TEXT _TEXT ends _DATA segment _DATA ends CONST segment __D4test12__ModuleInfoZ: db 004h,010h,000h,000h,000h,000h,000h,000h ;........ db 074h,065h,073h,074h,000h ;test. CONST ends _BSS segment _BSS ends FMB segment FMB ends FM segment dd offset FLAT:__D4test12__ModuleInfoZ FM ends FME segment FME ends _fun comdat assume CS:_fun ret _fun ends __D4test8__mixin43barUZv comdat assume CS:__D4test8__mixin43barUZv ret __D4test8__mixin43barUZv ends end ---

Right. We'll, that's obviously broken :)

Is it? Note the same behavior happens with: struct S { extern (C) void foo() { } } i.e. foo() gets mangled. There are two aspects of C mangling - one is the ABI, the other is the ABI. Using C mangling in a scope means there can be only one, so D just uses it to set the ABI. But there's another way you can do it: mixin template M() { export extern(C) pragma(mangle, "fun") void fun() {} } mixin M!(); which will work.

Why are you comparing it to a struct with methods? mixin is not a structured object with methods, and there's no reason to think those methods should receive a context object and be mangled accordingly. mixin inserts material into the lexical scope, which in this case is the global scope. It's the closest thing we have to a macro. I understand the workaround, but this is broken. It's caught a couple of people now. It's embarrassing when people see something that doesn't work with no reasonable explanation; we have a lot of that, and there's only so many occurrences people can handle before their confidence is lost. Easy low-hanging fruit like this is worth fixing, and we can spend our confidence erosion budget on the real issues.

> Using C mangling in a scope means there can be only one... Yes, and that's the entire point. That is how C is. Nobody would ever expect otherwise.

You can also use: enum M = "export extern(C) void fun() {}"; mixin(M);

It's just one declaration among a bunch of other stuff that is inserted into the global scope, and it's the only declaration that doesn't 'work'. It shouldn't be separated from all its associated declarations, and it's super weird that that's required. Mixin templates into the global scope appear in every single reflection-driven application I've ever written. One part of that generated code is the entrypoint function which registers the modules contents with the module loader. Seriously, just fix this. It's a bug.

Template mixins are meant to be included multiple times. That's why the scope is part of the mangled name.

A template mixin with an extern(C) function in it is obviously meant to be included at the global scope, why on earth would there be an extern(C) function in a template mixin otherwise? Especially an `extern` one. Nobody would be surprised by a link error complaining about multiple definitions of an extern(C) function; in fact, they'd be annoyed if it didn't. The pattern of a reflection template mixin placed at top-level is unbelievably common, and this code is common-sense for anyone trying to emit reflection-based data from a DLL. Lots of people don't use DLL's, so maybe it hasn't come up, but I use DLL's a lot. Seriously, the only conceivable useful thing you could do with an extern(C) function in a template mixin doesn't work. This is a bug. extern(C) means extern(C). There's no reason to re-interpret what extern(C) means in this one weird case. extern(C) specifies mangling and ABI, not mangling, or ABI, or maybe both.

(In reply to Manu from comment #9) > A template mixin with an extern(C) function in it is obviously meant to be > included at the global scope, why on earth would there be an extern(C) > function in a template mixin otherwise? Because there are uses for functions that follow the C ABI. Note that Microsoft C++ also distinguishes between C mangling and C ABI. Template mixins were designed so that boilerplate code can be inserted into many places.

One of the useful things about the current behavior would be mixing in a thing that needs a C callback function; the callback may be different for different instantiations.

And if there are extern(C) functions in the boilerplate, then ***THERE ARE EXTERN(C) FUNCTIONS IN THE BOILERPLATE***. That's literally what we are talking about! You can't change the names of extern(C) functions, or they don't work. There is absolutely no conceivable use for an extern(C) function that has been mangled with a bizarre scheme that you can't even declare. The function has been mangled, and there is literally no way to call it, or produce a C declaration that matches the function name, because you can't know what it is. I'm telling you what extern(C) functions in mixin templates are for, and you're telling me I'm wrong, you don't accept practical real-world software as evidence if its value, and instead insist that this bizarre choice of renaming the function to an un-knowable name is absolutely the proper thing to do?

One use case is to construct a C callback function, i.e. a pointer to a C function. Another is to use C variadic function parameters. Another is filling a table with C function pointers. Another is providing an alias to a C function template parameter. As I mentioned, the use case template mixins were designed for is for multiple inclusions of boilerplate. This won't work if the declarations conflict. Besides, I've provided two ways you can get it to work for your case. Since you are only instantiating a mixin template exactly once, those ways will work. It's why there is a pragma(mangle) feature. I suspect you're not manually writing those template mixins, but generating them. Generating a pragma(mangle) in them should not be a problem. The string mixin shouldn't be a problem for your case, either. At least, I don't understand why these would be problems.

I use those workarounds; but I still have to explain why it's broken to people. Those cases you describe, which are all cases of effectively anonymous functions, should be the ones that use pragma(mangle), you can generate any string you like and stuff it in there, the name is totally irrelevant. But you're telling that I should generate the proper name in the default case, which is the only case where the name IS relevant. C functions have naming requirements too; C functions need to have an underscore, or not, depending on the ABI. I shouldn't have to implement that logic to emit a C function name with or without underscore. The case where extern(C) doesn't mean extern(C) (your proposed cases where you want to emit many functions, which is distinctly NOT C) are the ones that should be doing special work; it should require effort to violate the meaning of extern(C), not the other way around. I also bet there are precisely zero use cases as you describe in existence.

Nice to see an explanation for this. > You can also use: > enum M = "export extern(C) void fun() {}"; > mixin(M); We do use the string mixin workaround for years, it hasn't been a huge problem, just a minor surprise. Meh.

Callbacks are more common than reflection-based dll exports. For some reason C++ doesn't allow nested extern "C" functions and they all get mangled.

Also wouldn't you want to export them without the leading underscore?

> Callbacks are more common than reflection-based dll exports. This is about what extern(C) means. mixin is supposed to inject code into the lexical scope it's mixed into. If you mixin to the global scope, it should be in the global scope, and there's nothing to mangle with. What we've learned here is that mixin is like some form of namespace, and that's NOT written on the tin, and definitely not what I have ever wanted. I bet almost nobody knows this, and they craft their mixins to work assuming they that they are inserted into the scope they're mixed into. This is also the part that doesn't make sense, an extern(C) function at global scope should be an extern(C) function, even if you want to make the rule that a nested one would be mangled somehow (since a nexted extern(C) function is kinda invalid anyway), the global scoped function should be as you expect. Callbacks nested in some scope are the thing that's distinctly not-C, and in that scenario the name is effectively anonymous anyway, it could be anything. That should be where you use the pragma(mangle) trick, with any string you like, because it doesn't matter; `pragma(mangle, makeAnonName())`. extern(C) should mean extern(C), and this case of "I half-want extern(C)" can be deliberately achieved with pragma(mangle). > Also wouldn't you want to export them without the leading underscore? It depends on the ABI. To use pragma(mangle), you need to determinate the ABI details and mangle with or without the underscore accordingly.

(In reply to Manu from comment #18) > What we've learned here is that mixin is like some form of namespace, and > that's NOT written on the tin, and definitely not what I have ever wanted. That's true indeed for template mixins. Not for string mixins btw, which is why extern(C) in string mixins does work like you want. "The declarations in a mixin are placed in a nested scope and then ‘imported’ into the surrounding scope." https://dlang.org/spec/template-mixin.html#mixin_scope So this is not 'one weird case' where extern(C) works differently, it's really that the rule shouldn't require the declaration to _be_ in global scope, but _accessible_ from global scope, because then a user may want to call it from C. Also, the spec says "C functions cannot be overloaded with another C function with the same name." (https://dlang.org/spec/interfaceToC.html), but with template mixins you can overload extern(C) functions currently. If you need extern(C) functions for a pointer table, you can simply do: ``` mixin template foo() { alias CFunc = extern(C) void function(int); CFunc func = (int a) {printf("%d\n", a);}; } ``` No need to give it a D-mangled name when it's not supposed to be exposed. > I bet almost nobody knows this, and they craft their mixins to work assuming > they that they are inserted into the scope they're mixed into. It does tend to be a surprise indeed. I first learned this when I wanted to mixin common opBinary functions and found that the specialized opBinary completely shadowed the mixed in ones. An alias or wrapper function needed to be added to make it work. It might be too late to change mixin template behavior though.

(In reply to anonymous4 from comment #16) > Callbacks are more common than reflection-based dll exports. Dplug has them: https://github.com/AuburnSounds/Dplug/blob/5ad0e4c512299fe8ea811ef2b041829f22f02a51/client/dplug/client/dllmain.d#L59 Excel-d has them: https://github.com/kaleidicassociates/excel-d/blob/ea7eb514e39e91fbc637e71a90d72614698dac18/source/xlld/wrap/traits.d#L234 It even mentions how "This used to be a template mixin but even using a string mixin inside fails to actually make it an extern(C) function.". Can you give an example where the proposed behavior would be problematic?

(In reply to Walter Bright from comment #3) > But there's another way you can do it: > > mixin template M() > { > export extern(C) pragma(mangle, "fun") void fun() {} > } > mixin M!(); > > which will work. That works ad-hoc because C mangling is relatively easy, but it asks the user to maintain their own C mangling logic, while the compiler should (and does) provide that.

(In reply to Dennis from comment #20) > Can you give an example where the proposed behavior would be problematic? Any callback. E.g. https://docs.microsoft.com/en-us/previous-versions/windows/desktop/legacy/ms633573%28v=vs.85%29

(In reply to anonymous4 from comment #22) > (In reply to Dennis from comment #20) > > Can you give an example where the proposed behavior would be problematic? > Any callback. E.g. > https://docs.microsoft.com/en-us/previous-versions/windows/desktop/legacy/ > ms633573%28v=vs.85%29 We're specifically looking for a situation where: 1 you have a C-ABI callback-only function 2 but it's not anonymous 3 and it's in a mixin template 4 which is mixed in at global scope 5 and its name clashes with other extern(C) functions / you want to mix it in multiple times Has anyone ever made a mixin template that mixes in a non-anonymous WindowProc resulting in clashing names? So far this situation is only hypothetical, while there are numerous instances of users running into the situation where they want a C-mangled mixin template: DPlug, Excel-D, this issue, https://issues.dlang.org/show_bug.cgi?id=962 and https://issues.dlang.org/show_bug.cgi?id=12575.

AIU, Manu wants extern(C) to be mangled the same in all situations.

To be fair, what I think is that extern(C) should be mangled the same in all places that it's valid... Which is; global scope. extern(C) doesn't have any precedent outside of global scope, and I just don't care what it does in places where I don't consider it valid. That said, if I had to choose, I'd say be the same everywhere. The odd guy out should be the explicit one.

Bringing this over from my forum post. Here's two code snippets. export extern(C++) void fun() {} pragma( msg, fun.mangleof ); ---- mixin template M() { export extern(C++) void fun() {} } mixin M!(); pragma( msg, fun.mangleof ); ---- Both cases print the exact same mangle - _Z3funv on Linux, ?fun@@YAXXZ on Windows. This illustrates two things: 1) extern(C++) behavior inside mixins is correct. extern(C) behavior is not. 2) pragma( mangle ) is not a catch-all workaround if you are compiling for different architectures/operating systems with different ABIs The Wikipedia article on name mangling[1] goes in to detail on the messy history of mangling C functions in the Windows ecosystem, if you're worried that using C++ as a counter-point is irrelevant. [1] https://en.wikipedia.org/wiki/Name_mangling#C

@dkorpel created dlang/dmd pull request #10236 "Fix issue 20012 - export inside mixin doesn't seem to work" fixing this issue: - fix issue 20012 - export inside mixin doesn't seem to work https://github.com/dlang/dmd/pull/10236

dlang/dmd pull request #10236 "Fix issue 20012 - extern(C) functions inside template mixins are not mangled as C functions" was merged into master: - 9b867ff4cae18953c864a7d9568e41295f09456a by dkorpel: fix issue 20012 - export inside mixin doesn't seem to work https://github.com/dlang/dmd/pull/10236

dlang/druntime pull request #2905 "Remove obsolete pragma(mangle, ...) from entrypoint.d" was merged into master: - 008c8a5e3a31cae3f1d444a681b7b1814c824c7a by MoonlightSentinel: Remove obsolete pragma(mangle, ...) from entrypoint.d They served as a workaround for issue 20012 which was fixed in the meanwhile. https://issues.dlang.org/show_bug.cgi?id=20012 https://github.com/dlang/druntime/pull/2905