Bug 17712 – [REG 2.074] [LINK] Undefined reference to std.conv.toChars!(10, char, 1, uint).toChars(uint)

Reduced test (with two selective imports). --- struct Bytecode { uint data; } @trusted ctSub(U)(string format, U args) { import std.conv : to; foreach (i; format) return format~ to!string(args); return format; } struct CtContext { import std.uni : CodepointSet; CodepointSet[] charsets; string ctAtomCode(Bytecode[] ir) { string code; switch (code) { OrChar: code ~= ``; for (uint i ; i ;) code ~= ctSub(``, ir[i].data); charsets[ir[0].data].toSourceCode; break; default: assert(0); } return code; } } --- Causes link error. --- backtracking.o: In function `_D3std4conv17__T6toImplTAyaTkZ6toImplFNaNbNekkE3std5ascii10LetterCaseZAya': __main.d:(.text._D3std4conv17__T6toImplTAyaTkZ6toImplFNaNbNekkE3std5ascii10LetterCaseZAya+0x5e): undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result' backtracking.o: In function `_D3std5array96__T5arrayTS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZ5arrayFNaNbNfS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZAa': __main.d:(.text._D3std5array96__T5arrayTS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZ5arrayFNaNbNfS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZAa+0xd): undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result6lengthMFNaNbNdNiNfZm' __main.d:(.text._D3std5array96__T5arrayTS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZ5arrayFNaNbNfS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZAa+0x4f): undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5emptyMFNaNbNdNiNfZb' __main.d:(.text._D3std5array96__T5arrayTS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZ5arrayFNaNbNfS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZAa+0x5c): undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result5frontMFNaNbNdNiNfZa' __main.d:(.text._D3std5array96__T5arrayTS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZ5arrayFNaNbNfS3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6ResultZAa+0x9f): undefined reference to `_D3std4conv47__T7toCharsVii10TaVE3std5ascii10LetterCasei1TkZ7toCharsFNaNbNiNfkZ6Result8popFrontMFNaNbNiNfZv' collect2: error: ld returned 1 exit status Error: linker exited with status 1 --- Prettified. --- backtracking.o: In function `std.conv.toImpl!(immutable(char)[], uint).toImpl(uint, uint, std.ascii.LetterCase)': __main.d:(0x5e): undefined reference to `std.conv.toChars!(10, char, 1, uint).toChars(uint)' backtracking.o: In function `std.array.array!(std.conv.toChars!(10, char, 1, uint).toChars(uint).Result).array(std.conv.toChars!(10, char, 1, uint).toChars(uint).Result)': __main.d:(0xd): undefined reference to `std.conv.toChars!(10, char, 1, uint).toChars(uint).Result.length()' __main.d:(0x4f): undefined reference to `std.conv.toChars!(10, char, 1, uint).toChars(uint).Result.empty()' __main.d:(0x5c): undefined reference to `std.conv.toChars!(10, char, 1, uint).toChars(uint).Result.front()' __main.d:(0x9f): undefined reference to `std.conv.toChars!(10, char, 1, uint).toChars(uint).Result.popFront()' collect2: error: ld returned 1 exit status Error: linker exited with status 1 --- Does not happen with 2.073. --- $ dmd-2073 backtracking.d -v 2>&1 | grep toChars.Result.popFront function std.conv.toChars!(10, char, cast(LetterCase)true, uint).toChars.Result.popFront function std.conv.toChars!(16, char, cast(LetterCase)false, uint).toChars.Result.popFront function std.conv.toChars!(16, char, cast(LetterCase)true, uint).toChars.Result.popFront function std.conv.toChars!(2, char, cast(LetterCase)true, uint).toChars.Result.popFront function std.conv.toChars!(8, char, cast(LetterCase)true, uint).toChars.Result.popFront $ dmd-2074 backtracking.d -v 2>&1 | grep toChars.Result.popFront function std.conv.toChars!(16, char, cast(LetterCase)false, uint).toChars.Result.popFront function std.conv.toChars!(16, char, cast(LetterCase)true, uint).toChars.Result.popFront function std.conv.toChars!(2, char, cast(LetterCase)true, uint).toChars.Result.popFront function std.conv.toChars!(8, char, cast(LetterCase)true, uint).toChars.Result.popFront --- However when testing with gdc, all I updated was the library, and managed to reproduce the same thing, so a change in phobos exposed this.

This seems to be the breaking change: https://github.com/dlang/phobos/pull/5017

Definitely looks like a front-end bug to me.

(In reply to ZombineDev from comment #2) > Definitely looks like a front-end bug to me. The link error does not happen when compiling the test with -unittest. So the specific location to look would be file dtemplate.d, function appendToModuleMember. The change in phobos was enough to make the instantiation speculative, so its arguable who's at fault. The compiler may be working as intended, though admittedly the whole design around speculative templates is not really backed by any spec.

LDC ran into the same issue when moving to Phobos 2.074. I ended up switching from individual druntime/Phobos object file compilation to all-D-files-at-once compilation (like DMD), as that reduces the template instantiation culling [1]. See LDC PR [2]. This template culling, in combination with restructured Phobos, also leads to sometimes dramatic performance decreases due to less inlining potential with non-instantiated (culled) templates, rendering cross-module-inlining/LTO essential for best performance. See this (lenghty but interesting) LDC issue [3]. There's a link to a Weka.io-specific patch where they had to disable the culling due to linker errors. For LDC, I'm planning to allow the user to prevent template culling via a command-line switch, as the current implementation doesn't seem very mature. [1]: https://github.com/dlang/dmd/blob/v2.075.0/src/ddmd/dtemplate.d#L7197-L7205 [2]: https://github.com/ldc-developers/ldc/pull/2076#issuecomment-315175464 [3]: https://github.com/ldc-developers/ldc/issues/2168

> For LDC, I'm planning to allow the user to prevent template culling via a command-line switch, as the current implementation doesn't seem very mature. I think that's what dmd's -allinst switch does currently.

(In reply to ZombineDev from comment #5) > > For LDC, I'm planning to allow the user to prevent template culling via a command-line switch, as the current implementation doesn't seem very mature. > > I think that's what dmd's -allinst switch does currently. [LDC uses the same logic and command-line switch.] I find it quite hard to tell the exact difference when using `-allinst` (and whether that includes speculative ones), as it's an early return of that function and the remaining needsCodegen() logic isn't trivial; the interaction with the tnext chain doesn't make it any simpler unfortunately.

For reference: the missing symbol also causes a backend ICE for GCC <= 4.9: https://bugzilla.gdcproject.org/show_bug.cgi?id=157 This is fortunately of lower priority as an (unrelated) GCC change works around this problem in all new GCC versions. Fortunately we're already at GCC-7 released / GCC-8 in development :-)

There's something similar here : https://issues.dlang.org/show_bug.cgi?id=18234 Excepted that it started failing from 2.075, not 2.074

*** Issue 18234 has been marked as a duplicate of this issue. ***

(In reply to Iain Buclaw from comment #3) > The link error does not happen when compiling the test with -unittest. I just want to point out that using -unittest currently is the same as using -allinst.

interesting fact: The problem doesn't happen when std.conv.toChars template parameter `radix` is changed from `ubyte` to `int` or when it changed to an `alias radix = 10`

speculative could fail because it wrongly thinks that it will ALREADY be codegened. in toImpl (line 1394 of std.conv) we have --- return toChars!(10, EEType)(value + 0).array; --- but if we specifies the type of value: --- return toChars!(10, EEType, LetterCase.lower, typeof(value + 0))(value + 0).array; --- than the test case passes. Maybe IFTI fails/is wrong with "value + 0", trick to promote) and then it gives another template that is HIM codegened.

Other test case: --- import std.datetime; import std.typecons; import std.variant; Y a() { Y n = Y(Y[].init); n.get!(X[]); return n; } struct X { Y key; } struct Y { Algebraic!(Y[]) value_; this(T)(T value) { value_ = value; } bool opEquals(T)(T rhs) const { static if(is(Unqual!T == Y)) { return true; } else { return get!(T, No.x) == get!T; } } T get(T, Flag!"x" x = Yes.x)() const { return this[""].get!(T, x); } Y opIndex(T)(T index) const { const X[] pairs; if(pairs[0].key == index) { assert(0); } assert(0); } } void main(){} --- linker error without -allinst

This bug would be much easier to troubleshoot if it could be reduced to a couple of modules and *not* import any Phobos module. A -betterC reproduction would be even better. There's just too much noise when parsing and semantic-processing Phobos modules to see what's wrong and where.

Still reproducible on 2.081 release candidate (I've had to revert the commit again).

Commits pushed to master at https://github.com/dlang/phobos https://github.com/dlang/phobos/commit/2f20661b0e0e8072cb0c056caddc3a7ef3cfd76a Fix Issue 17712 - [REG 2.074] [LINK] Undefined reference to std.conv.toChars!(10, char, 1, uint).toChars(uint) https://github.com/dlang/phobos/commit/217c9af28f6fc9a4f8dc6f11d0eae0ed58805c43 Merge pull request #6659 from shove70/patch-1 Fix Issue 17712 - [REG 2.074] [LINK] Undefined reference to std.conv.toChars!(10, char, 1, uint).toChars(uint) merged-on-behalf-of: Nathan Sashihara <[email protected]>

I'm not too happy about closing this, as the underlying issue definitely isn't fixed. I bet https://github.com/ldc-developers/ldc/issues/2846 (not LDC-specific) is yet another instance of this template culling issue.

How hard would it be to have a @allinst function attribute, or better, a pragma(allinst) ? * -allinst is globally applied so not a ideal workaround. * workarounds like the one that's been merged can't be done each time the problem is encountered in phobos.

(In reply to kinke from comment #17) > I'm not too happy about closing this, as the underlying issue definitely > isn't fixed. Yes, and the workaround didn't even add a test case to prevent someone just changing std.conv.text to use toChars again. The toChars radix workaround (comment #11) would have had much lower impact. I haven't been able to reproduce the original bug with dmd v2.092.0, maybe I'm doing something wrong though.

@ntrel created dlang/dmd pull request #11300 "Fix Issue #17712 - Add test cases" fixing this issue: - Fix Issue #17712 - Add test cases https://github.com/dlang/dmd/pull/11300

Reduced test case without phobos that still fails with current dmd: ============================================= // stdx/conv.d: void to(T)(uint args) { toChars!10(args); } alias CodepointSet = InversionList!(); struct InversionList() { void toSourceCode() { toChars(0u); } } void toChars(ubyte radix = 10, T)(T) {} ============================================= // test.d: void main() { import stdx.conv; to!string(0u); CodepointSet().toSourceCode(); } ============================================= dmd -c stdx/conv.d dmd test.d conv.o

(In reply to moonlightsentinel from comment #21) > Reduced test case without phobos that still fails with current dmd: > > ============================================= > // stdx/conv.d: > > void to(T)(uint args) > { > toChars!10(args); > } > > alias CodepointSet = InversionList!(); > > struct InversionList() > { > void toSourceCode() > { > toChars(0u); > } > } > > void toChars(ubyte radix = 10, T)(T) {} > > ============================================= > // test.d: > > void main() > { > import stdx.conv; > to!string(0u); > CodepointSet().toSourceCode(); > } > ============================================= > > dmd -c stdx/conv.d > dmd test.d conv.o I cannot reproduce this.