Bug 3274 – dmd fails to emit code for templates into object file if several files are compiled at once

If you compile several files at once, and the files instantiate the same template, dmd outputs the code for that template only into one object file. This breaks incremental compilation with tools like rebuild/dsss. ldc doesn't suffer from this bug, and outputs the template code into all corresponding object files. Reproduction of that bug: First put the files listed at the end of the bug report into a.d, b.d and c.d. Then execute these steps (order of files is important): 1. dmd a.d b.d c.d -c -version=XX 2. dmd a.d -c (without version XX -> like a change to the source code; because b.d and c.d don't depend from a.d, you shouldn't need to recompile it.) 3. dmd a.o b.o c.o Step 3 gives the error: b.o: In function `_D1b1xFZv': b.d:(.text._D1b1xFZv+0x4): undefined reference to `_D1c13__T6from_cTiZ6from_cFZv' collect2: ld returned 1 exit status --- errorlevel 1 If Step 2. is left out, the error doesn't happen. If you compile all files separately in Step 1, the error doesn't happen. -- file a.d: module a; import b; import c; void main() { version (XX) { from_c!(int)(); } } -- file b.d: module b; import c; void x() { from_c!(int)(); } -- file c.d: module c; void from_c(T)() { }

The compiler does this as a compile performance optimization. Otherwise, template instantiations have to be done over and over, filling the object files, and then the linker has to remove the duplicates. The workaround for an incremental build system is straightforward - put only one source module on a command to dmd at a time.

It seems to work correctly with DMD v2.062 at least on Fedora 64 bit. Can anyone confirm this?

(In reply to comment #1) > The compiler does this as a compile performance optimization. Otherwise, > template instantiations have to be done over and over, filling the object > files, and then the linker has to remove the duplicates. > It remains an incorrect optimization that just works most of the time. > The workaround for an incremental build system is straightforward - put only > one source module on a command to dmd at a time. The performance impact of this is also huge. The only reason to do this in the first place is to avoid duplicate parsing and semantic. However we can't make any assumptions how the generated object files are linked therefor each of them needs to get the required TypeInfo, templates et.al. To put this into perspective, copying is only required when building multiple object files. It is not required when building an executable, a shared library or a single object (which may contain multiple modules, i.e. -c -of). Maybe we can move the copying more to the backend so that we don't have to rerun IRgen and codegen.

One idea to reduce the cost of copying is partial linking. AFAIU that would allow us to generate a single temporary object containing the template/TypeInfo instantiation and link it into every object file that references it. ld -r -o combined1.o template.o module1.o mv combined1.o module1.o ld -r -o combined2.o template.o module2.o mv combined2.o module2.o http://sourceware.org/binutils/docs-2.23.1/ld/Options.html#index-partial-link-88 Though I'm not sure how portable this or an equivalent dmd backend functionality would be.

*** Issue 8769 has been marked as a duplicate of this issue. ***