We produce a function prototype in a .di file using some meta
test.di
-------
module test;
template Test()
{
import std.range, std.algorithm;
enum Test = "void test_func(" ~
zip(["int","int"], ["a","b"])
.map!(tuple => tuple[0] ~ " " ~ tuple[1])
.reduce!((a, b) => a ~ ", " ~ b) ~
");";
}
mixin(Test!()); // <- Mixes in: void test_func(int a, int b);
-------
main.d
------
import test;
int main()
{
test_func(1, 2);
return 0;
}
------
I expect compiling this example to result in a single link error to the unresolved external test_func().
But we also see "error LNK2001: unresolved external symbol _D3std5range__T3ZipTAAyaTQfZQn6__initZ"
Calling zip() at CTFE results in an additional unexpected link error.
Using a function in CTFE this way shouldn't affect the symbol table, or add any additional link references.
Comment #1 by hsteoh — 2018-04-17T22:53:42Z
AIUI, all template functions that are instantiated will end up in the object file, even if they are only used during CTFE (i.e. the instantiation is triggered by CTFE but not anywhere else). This is a problem that's been bothering me too, which has led me to insert `if (__ctfe)` blocks into my CTFE functions. But if you're using Phobos functions, then you're out of luck.
I agree that the compiler should have some way to track whether a particular template function / instantiation / etc., is only referenced by CTFE code, so that said functions are omitted from the object file when emitting code. Possibly something like a boolean flag in the template symbol to indicate whether the symbol is referenced by runtime constructs.
I suspect, though, that this is far from trivial to implement, because code that gets run by CTFE is essentially just one step away from being emitted as object code, and for all intents and purposes is considered as "runtime" code by many parts of the compiler. So you get tricky things like whether a function called by a CTFE function should be marked as CTFE-only or should be included in the object file (since that function call would have been resolved before the CTFE engine even sees it, so it would already have been considered as "runtime", even if in actuality it will only ever be called from CTFE).
Still, it's a nice-to-have.
Comment #2 by hsteoh — 2018-04-17T22:54:43Z
P.S. and the `if (__ctfe)` hack I use only saves on executable size, it doesn't suppress the symbol itself from appearing in the object file, so it's probably not good enough for your use case.
Comment #3 by turkeyman — 2018-04-17T22:59:57Z
Well in this case, the symbol is emit in a .di file, which means it's never written to any object file; hence the unresolved external...
But that's actually kind-of irrelevant. The big question I have here is, where is the link error coming from?! Why is there a reference in main.obj to Zip!(...)'s init value? Where could that symbol reference possibly be coming from? There's no runtime calls to zip().
I don't understand how the link error is even emerging, because there should be nothing making such a reference.
Comment #4 by issues.dlang — 2018-04-18T00:43:41Z
If the .di file is being used, then the code that's importing it would expect the symbol, but if the object file is then generated from the .d file, the symbol isn't there, thus I would expect a linker error. AFAIK, the only way around that would be if the compiler were smart enough to only use the symbol for as long as necessary to do the CTFE call, because it's only used during CTFE, and I don't think that it's that smart, much as I think that we can all agree that we want it to be that smart.
I'd suggest that you provide the actual compilation commands that you're using so that it's 100% clear how to reproduce what you're doing.
Comment #5 by turkeyman — 2018-04-18T02:08:30Z
> dmd -m64 main.d
Emits:
main.obj : error LNK2019: unresolved external symbol _D4test9test_funcFiiZv referenced in function _Dmain (expected)
main.obj : error LNK2001: unresolved external symbol _D3std5range__T3ZipTAAyaTQfZQn6__initZ (WTF?)
I think you've missed my point, there is NO REFERENCE to Zip!().init in main.d, at least, there shouldn't be... so why the link error?
Where is the reference coming from?
Comment #6 by issues.dlang — 2018-04-18T03:03:25Z
(In reply to Manu from comment #5)
> I think you've missed my point, there is NO REFERENCE to Zip!().init in
> main.d, at least, there shouldn't be... so why the link error?
> Where is the reference coming from?
You imported test.di. test.di references Zip!().init, and it references it when creating the function that you're calling from main.d. So, it's being used when generating main.d even if main.d doesn't use it directly. I'd guess that the problem relates to that. It's likely inserting the symbol when it imports test.di. I don't see what else the problem could be. Also, remember that you're actually generating template instantations with that code. It's not just referencing existing code like it would with a normal function. Template instantations don't live with the module that they come from in the same way that other functions do, because they can't, since they're not compiled with the module. I'd guess that that's part of the problem and that it results in the template instantiations being inserted into the resultant binary.
Curiously, if I compile your example on FreeBSD (with clang as the C/C++ compiler, though I think that it's actually using GNU's linker), I only get an undefined reference for `_D4test9test_funcFiiZv', so it appears that the problem with Zip may be system-dependent. Maybe one linker ignores it because it isn't used directly in main.d, and the other doesn't? I don't know. Certainly, the fact that it appears to differ from system to system doesn't give me a warm, fuzzy feeling.
Comment #7 by turkeyman — 2018-04-18T03:57:36Z
Orly? Doesn't happen for you? That's even worse.
Just the presence of the declaration shouldn't result in a link error.
You can declare all the undefined functions you like, you don't get link errors unless there is a reference to them that the linker tries (and can't) resolve.
I don't understand what reference there could possibly be to Zip!().init, even if it was generated and emit to the symbol table when it was instantiated by CTFE.
Your hunch that it's emit into some sort of template-instantiation namespace rather than into a module is probably on the money. It smells like the problem could be tangled in that complexity... but at the end of the day, I just can't imagine where the reference is that the linker could be upset about.
Anyway, this was discovered by a colleague at Blizzard who's exploring D. It would certainly be reassuring if we could fix it promptly.
Comment #8 by issues.dlang — 2018-04-18T04:06:18Z
Well, regardless of what's actually causing the problem, I completely agree that it should be fixed. However, not being a compiler dev, I have no clue how likely that is to be quick or easy.