Bug 10920 – template instantiation order dependent link failure problem

Current dmd has an essential problem around template instantiation. It is rely on the semantic order, and would cause "undefined symbol" error in link phase. Reduced code: // foo.d import bar; void main() { BitArray ba; version(A) ba.toString(); FormatSpec!char fs; fs.func(); } // bar.d struct FormatSpec(C) { void func() {} } struct BitArray { auto toString() { FormatSpec!char fs; fs.func(); } } Command line: $ dmd foo.d --> OK $ dmd -version=A foo.d OPTLINK (R) for Win32 Release 8.00.12 Copyright (C) Digital Mars 1989-2010 All rights reserved. http://www.digitalmars.com/ctg/optlink.html bar.obj(bar) Error 42: Symbol Undefined _D3baz8BitArray8toStringMFZv --- errorlevel 1 --> NG When -version=A switch is not specified, FormatSpec!char is instantiated on the function main in foo module at first. It would emit the the instance in the member of foo module, then codegen phase would output the generated code correctly. When -version=A switch is specified, the ba.toString() call runs the semantic3 of the function BitArray.toString first. It would instantiate FormatSpec!char in bar module, then emit the instance in the member of bar module. This is a serious issue. If modules are separately compiled, and templates are already instantiated in the imported module, it could cause the undefined symbol errors.

I think this is one of the cause of bug 10631. And, unfortunately, long term bug fix around template opEquals/opAssign/opCmp - e,g, fixing bug 3789, bug 4424, bug 3659, and others - had increased the criticalness of this issue.

> ...and others - had increased the criticalness of this issue. Filed particualr regression Issue 11114.

I think it was already discussed in that Walters pull request and rejected as invalid. If `foo.d` imports `bar.d` it MUST also compile `bar.d` too and link into the final application. In provided snippet `FormatSpec!char` should always be expected to be in `bar.d` during separate compilation. It may also be _additionally_ emitted to `foo.d` as a weak symbol if `foo.d` code flow runs semantic on it first, but that will be taken care of by linker. D does not have headers, every single module is expected to be compiled and linked.

(In reply to comment #3) > I think it was already discussed in that Walters pull request and rejected as > invalid. If `foo.d` imports `bar.d` it MUST also compile `bar.d` too and link > into the final application. My question and Walter's reply: https://github.com/D-Programming-Language/dmd/pull/2550#issuecomment-24296228 https://github.com/D-Programming-Language/dmd/pull/2550#issuecomment-24297212 But I still worries that the current design is difficult to understand.

(In reply to comment #4) > (In reply to comment #3) > > I think it was already discussed in that Walters pull request and rejected as > > invalid. If `foo.d` imports `bar.d` it MUST also compile `bar.d` too and link > > into the final application. > > My question and Walter's reply: > https://github.com/D-Programming-Language/dmd/pull/2550#issuecomment-24296228 > https://github.com/D-Programming-Language/dmd/pull/2550#issuecomment-24297212 > > But I still worries that the current design is difficult to understand. Yeah, that is exactly the comments I was referring to, thanks. And I completely agree with Walter here - this is difficult to understand only if you think about modules in a similar way as C headers, but those are not. Average programmer should care about internal design as much as he cares about exact optimization algorithms in backend. Omitting some modules during compilation is illegal and likely to break the program in some way (not only current one), that should be told explicitly in spec if it is not already.

Sorry, the OP code was not correct. Precise test case is: // foo.d import bar; void main() { BitArray ba; version(A) pragma(msg, typeof(ba.toString)); FormatSpec!char fs; fs.func(); } // bar.d (same with in comment #0) Command line: $ dmd foo.d --> OK $ dmd -version=A foo.d void() OPTLINK (R) for Win32 Release 8.00.15 Copyright (C) Digital Mars 1989-2013 All rights reserved. http://www.digitalmars.com/ctg/optlink.html foo.obj(foo) Error 42: Symbol Undefined _D3bar18__T10FormatSpecTaZ10FormatSpec4funcMFNaNbNiNfZv --- errorlevel 1 --> NG FormatSpec!char is instantiated in both main() and BitArray.toString(). When -version=A specified, the first instantiation happens in toString(), and its codegen is wrongly judged to be unnecessary. And the second instantiation is ignored and link-failure occurs. It's definitely an issue that depends on the semantic analysis order.

Commits pushed to master at https://github.com/D-Programming-Language/dmd https://github.com/D-Programming-Language/dmd/commit/0b695233589c6124a359645137aea803e2fe84e4 fix Issue 10920 - template instantiation order dependent link failure problem https://github.com/D-Programming-Language/dmd/commit/c4387428c034ae83fa2504379c68ed3ac354d58f Merge pull request #4384 from 9rnsr/fix2644 Issue 2644 & 2500 & 10920 - Unresolved template reference

Commits pushed to https://github.com/D-Programming-Language/dmd https://github.com/D-Programming-Language/dmd/commit/0b695233589c6124a359645137aea803e2fe84e4 fix Issue 10920 - template instantiation order dependent link failure problem https://github.com/D-Programming-Language/dmd/commit/c4387428c034ae83fa2504379c68ed3ac354d58f Merge pull request #4384 from 9rnsr/fix2644