Bug 2962 – ICE(glue.c) or bad codegen passing variable as template value parameter

Created attachment 358 Compile and run this file While porting a library from D1 to D2 I encountered the following DMD error: dmd: glue.c:652: virtual void FuncDeclaration::toObjFile(int): Assertion `!v->csym' failed. Aborted It's hard to say what causes the error, but I've been able to narrow it down to a point where almost any change I make makes the error disappear. The source files are attached. (There's four of them, but they are very short.) To reproduce the error, compile and run the program with rdmd: rdmd moduleA.d Strangely enough, running DMD directly does not reproduce the error: dmd moduleA.d moduleB.d moduleC.d moduleD.d ./moduleA For some changes a runtime bug is introduced instead; see the commented section in moduleC.d.

Created attachment 359 moduleB: Imported by moduleA

Created attachment 360 moduleC: imported by moduleB

Created attachment 361 moduleD: imported by moduleC

Here is a simpler test case for what I think is the same issue: -----8<----testa.d----- import testb; int foo() { return bar(0); } -----8<----testa.d----- -----8<----testb.d----- T bar(T)(T x) { return baz!(T, x)(); } T baz(T, T x)() { return x; } -----8<----testb.d----- Compile order matters: > dmd -c testb.d testa.d Assertion failure: '!v->csym' on line 563 in file 'glue.c' abnormal program termination > dmd -c testa.d testb.d > Tested this with DMD 1.030, 1.033, 1.041, 1.042, 1.046, 2.026, 2.027 and 2.031, with exactly the same results.

Sorry, this ICE is definitely on *invalid* code since the baz template is being parametrized on the bar's run-time argument.

Fundamentally this is an accepts-invalid bug, it shouldn't reach the code generation stage where the ICE occurs. Here's a test case which shouldn't compile. There's a chance this could be a regression. --- T bar(T)(T y) { return baz!(T, y)(); } T baz(T, T z)() { return z*z; } void main() { assert(bar(4)!=0); }

It's definitely not a recent regression, D1.020 behaved the same way. Probably the same as 2733, (but please DO NOT mark this as a duplicate, this is a much better bug report). Even smaller test case: void baz(int z)() {} void main() { int x; baz!(x)(); }

Actually I'm not sure if this a bug or an easter egg. The current behaviour of template value parameters is: if it is a compile-time constant, instantiate the template as a value. If it is a variable, instantiate it as an alias parameter. That's actually quite cool.

By the way, the example Lars posted is not as obviously invalid, at least to me. He passes a local variable as a template alias parameter. Docs say that "local names" can be used as template alias parameters. This actually works correctly: import std.stdio; void main() { foo(1); } void foo(int a) { bar!(a)(); writefln(a); } void bar(alias var)() { var = 2; } Prints 2. This works even if bar is defined in a different module.

(In reply to comment #8) > Actually I'm not sure if this a bug or an easter egg. The current behaviour of > template value parameters is: if it is a compile-time constant, instantiate the > template as a value. If it is a variable, instantiate it as an alias parameter. > > That's actually quite cool. Local variables have been passable as alias parameters for quite some time now. :)

(In reply to comment #10) > (In reply to comment #8) > > Actually I'm not sure if this a bug or an easter egg. The current behaviour of > > template value parameters is: if it is a compile-time constant, instantiate the > > template as a value. If it is a variable, instantiate it as an alias parameter. > > > > That's actually quite cool. > > Local variables have been passable as alias parameters for quite some time now. > :) Yes, but if you declare a template parameter as 'alias', it can't accept literals. You can actually achieve the same effect with a tuple parameter, so this behaviour should probably disappear.

What I find very strange here is that when I compile my test case with DMD 2.031 specifying just the four attached files, it works. And it should -- I think the code is valid according to the spec of both D1 and D2. lars@neutrino:~/tmp/bug2962$ dmd moduleA.d moduleB.d moduleC.d moduleD.d lars@neutrino:~/tmp/bug2962$ ./moduleA 1 However, when I compile it like rdmd does, specifying every module in Phobos on the command line, it fails: lars@neutrino:~/tmp/bug2962$ dmd 'moduleA.d' 'moduleC.d' '/usr/local/include/d/druntime/core/sys/posix/config.d' '/usr/local/include/d/druntime/core/stdc/stdio.d' '/usr/local/include/d/druntime/core/sys/posix/fcntl.d' [...] dmd: glue.c:658: virtual void FuncDeclaration::toObjFile(int): Assertion `!v->csym' failed. (Run rdmd with --dry-run to see the full list of files it passes to DMD.)

(In reply to comment #12) > What I find very strange here is that when I compile my test case with DMD > 2.031 specifying just the four attached files, it works. And it should -- I > think the code is valid according to the spec of both D1 and D2. > > lars@neutrino:~/tmp/bug2962$ dmd moduleA.d moduleB.d moduleC.d moduleD.d Try dmd moduleA.d moduleC.d moduleB.d moduleD.d

Oh, my. :) Didn't even cross my mind that the order of the files matter. But my example IS valid code, right?

I do think it's valid, see my comment #9. According to the obj2asm, when you pass a local var as an alias parameter to a template: void foo() { int i; bar!(i)(); } void bar(alias var)() { var = 1; } compiler rewrites it as void foo() { int i; void bar() { i = 1; } bar(); } Probably this rewrite is not always possible, hence the assertion failure.

Seems like this bug is an ice-on-VALID after all. All our examples boil down to passing local variables as alias template arguments. Also it seems like a nice candidate for the issue 340 list. Don, what do you think?

(In reply to comment #16) > Seems like this bug is an ice-on-VALID after all. All our examples boil down > to passing local variables as alias template arguments. Also it seems like a > nice candidate for the issue 340 list. Don, what do you think? Could be. I'm busy on other stuff right now, so haven't had a good look at it. Are you able to simplify the valid code example? It's worth all bug reporters knowing that the first step to fixing a bug is to minimize the test case that's in Bugzilla. It helps a _lot_. I found that 90% of the bugs in bugzilla can be minimized further.

I don't have a patch, but it's easy to at least get the line number into the ICE message. glue.c line 658: if (v->csym) error("Bugzilla 2962 - Internal Compiler Error"); Doing this would greatly reduce frustration, and probably allow us to close bug #3283. I'm sure it's a duplicate.

*** Issue 3283 has been marked as a duplicate of this issue. ***

Here's a reduced test case for the ice-on-valid case. dmd moduleC.d moduleA.d The instantiation of funcD inside funcC is failing, if funcC is only instantiated from another module. Some kind of instantiation order problem (semantic not run, perhaps). This is definitely valid code. Quite probably related to the other template alias ICE and bad codegen bugs.(eg, bug 3293, bug 2325, bug 2845, ...) moduleA.d ========== import moduleC; void main() { funcC!(bool)(1.0); } ===== moduleC.d ======= void funcD(alias x)() { assert(x==1.0); } void funcC(T)(double a){ // Case 1: ICE(glue.c) funcD!(a)(); // Case 2: wrong code double b = 1.0; funcD!(b)(); }

This is really tough, it's an order-of-evaluation issue. When generating the code for a template, which has a local variable as an alias parameter, the alias parameter MUST be created before the code for template is. But I have no idea how the order of code generation is supposed to be enforced. It seems to always get it right if everything is in the same file.

Related SVN commit: http://www.dsource.org/projects/dmd/changeset/240

I can confirm that this bug is dependent on the order of code generation. I just ran into this on a project I'm working on and it seems that whether this bug is exposed or not depends on the order in which I pass the files to DMD. Some permutations result in my project compiling and linking w/o errors. Others result in this bug being exposed.

Phobos unittest is broken. I tried to build dmd, druntime, and phobos of trunk. You should notice -unittest below. c:\d\svnproj\phobos_trunk\phobos\std>dmd -c -of_ -unittest algorithm.d stdio.d ... std.algorithm.canFindSorted is scheduled for deprecation. Use std.range.SortedR ange.canFind instead. std.algorithm.lowerBound is scheduled for deprecation. Use std.range.SortedRang e.lowerBound instead. std.algorithm.upperBound is scheduled for deprecation. Use std.range.SortedRang e.upperBound instead. std.algorithm.equalRange is scheduled for deprecation. Use std.range.SortedRang e.equalRange instead. C:\D\dmd2\windows\bin\..\..\src\phobos\std\conv.d(1263): Error: function std.con v.parse!(float,LockingTextReader).parse compiler error, parameter 'p', bugzilla 2962? Assertion failure: '0' on line 729 in file 'glue.c' abnormal program termination

Sorry, you will encounter this issue when dmd and phobos are debug build. For dmd, you will build dmd with "make -f win32.mak" and "DEBUG=-g -D -DUNITTEST -L/detailedmap". For phobos, "you will build phobos with "make -f win32.mak" and "DFLAGS=-unittest -g -d -debug -L/detailedmap". If you do so, you will get the following result. C:\d\svnproj\phobos_trunk\phobos\std>dmd -c -of_ -unittest algorithm.d stdio.d 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2 : 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2 : 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2 : 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: std.algorithm.canFindSorted is scheduled for deprecation. Use std.range.Sor tedRange.canFind instead. std.algorithm.lowerBound is scheduled for deprecation. Use std.range.SortedRang e.lowerBound instead. std.algorithm.upperBound is scheduled for deprecation. Use std.range.SortedRang e.upperBound instead. std.algorithm.equalRange is scheduled for deprecation. Use std.range.SortedRang e.equalRange instead. 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2: 2 : 2: 2: C:\D\dmd2\windows\bin\..\..\src\phobos\std\conv.d(1263): Error: function std.conv.parse!(float,LockingTextReader).parse compiler error, parameter 'p', b ugzilla 2962? assert glue.c(729) 0 <---- AND HALT!

(In reply to comment #25) > Sorry, you will encounter this issue when dmd and phobos are debug build. > For dmd, you will build dmd with "make -f win32.mak" and "DEBUG=-g -D > -DUNITTEST -L/detailedmap". No problem, I was able to reproduce it from your first comment.

A simpler command line is this: /dmd/src/phobos/std> dmd -c -unittest conv.d stdio.d The unittest which it's failing in, is in stdio.d, line 1630: unittest { float f; if (false) readf("%s", &f); } Which just shows how nasty this bug is.

(In reply to comment #27) > A simpler command line is this: > /dmd/src/phobos/std> dmd -c -unittest conv.d stdio.d > The unittest which it's failing in, is in stdio.d, line 1630: > > unittest > { > float f; > if (false) readf("%s", &f); > } > Which just shows how nasty this bug is. This is a progress report. failed: /dmd/src/phobos/std dmd -c -unittest conv.d stdio.d succeeded: /dmd/src/phobos/std dmd -c -unittest stdio.d conv.d

A simpler sample is this: // module a import b; void main() { foo!int(0); } // module b void foo(T)(T p) { void inner(U)() { auto p2 = p; } inner!int(); } C:\Users\haru\Desktop\c>dmd b a b.d(1): Error: function b.foo!(int).foo compiler error, parameter 'p', bugzilla 2962? assert glue.c(729) 0 <--------------------------------- HALT However, the following result is success. C:\Users\haru\Desktop\c>dmd a b Max # of fixups = 8

I've also hit this issue today after updating to the latest dmd and runtime from github. As there is no hint where the problem is in my project, I took a look at the last test case, and figured out a few things: - the template instantiation for foo is added to module a, but the instantiation of inner is added to module b. - the error is triggered because the code for the inner function is generated before the code for the outer function. Hence the reliance on the order of modules on the command line. - that made me think, that it might be ok to just skip the test, but the following code shows, that it does not work because the inner function needs the stack offset which is only calculated when the outer function is generated: // module b T foo(T)(T p, int q) { T inner(U)() { return p; } return inner!int(); } // module a import b; void main() { assert(foo!int(5, 2) == 5); } fails for "dmd b.a b.d", but not for "dmd a.d b.d". Is there a good reason why the inner function is generated into a different module than the outer function?

Investigating this a little further, I noticed that when compiling "dmd b a" the problem seems to be that inner!int is *parsed* before the invocation of the outer foo, which causes a TemplateInstance to be created. This later causes the code generation of inner to come first, making the compiler bail out, because the reference to the parameter of the outer function is not known yet. As a workaround it seems to work if you put a reference to foo!int where the parser sees it before inner!int, for example into another file c.d: module c; static if(__traits(compiles,foo!int))){} and the compile it with "dmd c b a".

Ouch. Hit this when compiling SDC ( https://github.com/bhelyer/SDC ) with 2.052. I'm stuck on 2.051 for the time being.

dmd v2.052, Ubuntu, quite short example of this bug: $ cat main.d import std.stdio; void main(){} $ cat utils.d import std.json; auto val = &parseJSON!string; $ dmd main.d utils.d /usr/include/d/dmd/phobos/std/conv.d(1301): Error: function std.conv.parse!(real,string).parse compiler error, parameter 'p', bugzilla 2962? dmd: glue.c:734: virtual void FuncDeclaration::toObjFile(int): Assertion `0' failed. Aborted (core dumped)

(In reply to comment #33) > dmd v2.052, Ubuntu, quite short example of this bug: <snip> No, that's a long example -- it imports from Phobos. Test cases for compiler bugs which import from Phobos are not completely reduced. The examples in comments 20 and 29 are minimal test cases.

*** Issue 7323 has been marked as a duplicate of this issue. ***

cat > a.d << CODE import b; alias foo!() fooi; CODE cat > b.d << CODE void foo()(int p) { int inner()() { return p; } alias inner!() finner; } CODE dmd -c b a -------- Slightly more reduced test case. There are two things that should get fixed. - We push template instances into the wrong object/module. The module a template instance is attached to is found by following the importedFrom chain. The head of the chain should be the template declaration module not the instantiation module. By doing so we guarantee that all instances remain ordered and are bundled with their dependencies. - VarDeclaration::toSymbol creates csym lazily. I don't see any reason why this shouldn't apply to parameters as well, e.g. replacing the 'p' parameter with a local variable fixes the bug. So we should just call v->toSymbol() and remove the assertion.

Commits pushed to master at https://github.com/D-Programming-Language/phobos https://github.com/D-Programming-Language/phobos/commit/47f5929059811b4b0d2d35c7833cabd7db490c8e workaround Bug 2962 - ICE with nested template functions - there is no need to templatize bailOut https://github.com/D-Programming-Language/phobos/commit/717bd999ba0427d5bd884cded218355f735c5674 Merge pull request #505 from dawgfoto/workaround2962 workaround Bug 2962

(In reply to comment #36) > cat > a.d << CODE > import b; > alias foo!() fooi; > CODE > > cat > b.d << CODE > void foo()(int p) > { > int inner()() { return p; } > alias inner!() finner; > } > CODE > > dmd -c b a > > -------- > > Slightly more reduced test case. > > There are two things that should get fixed. > > - We push template instances into the wrong object/module. > The module a template instance is attached to is found > by following the importedFrom chain. The head of the chain > should be the template declaration module not the instantiation > module. By doing so we guarantee that all instances remain > ordered and are bundled with their dependencies. It's interesting how closely related that issue is to this one I posted yesterday: https://github.com/D-Programming-Language/dmd/pull/824 Fundamentally we need to sort out which module owns a function. I'm not sure where the code belongs. > > - VarDeclaration::toSymbol creates csym lazily. > I don't see any reason why this shouldn't apply to > parameters as well, e.g. replacing the 'p' parameter > with a local variable fixes the bug. So we should > just call v->toSymbol() and remove the assertion.

Commit pushed to dmd-1.x at https://github.com/D-Programming-Language/dmd https://github.com/D-Programming-Language/dmd/commit/d5a33a1cbeedec2ea65ba480636caaf78ca05117 fix Issue 2962 - ICE(glue.c) or bad codegen passing variable as template value parameter

Commit pushed to master at https://github.com/D-Programming-Language/dmd https://github.com/D-Programming-Language/dmd/commit/900c537038932435eaebdf7c0f80926e0bd8f2f5 fix Issue 2962 - ICE(glue.c) or bad codegen passing variable as template value parameter

Commit pushed to master at https://github.com/D-Programming-Language/dmd https://github.com/D-Programming-Language/dmd/commit/816219255151628cfc2a8dc279f603420b7d9312 Test cases for bug 2962 ICE(glue.c) with variable as template value parameter

The following code (two modules) shows that this bug is not fixed yet. module foo; void baz(alias pred)() { pred(0); } --- module bar; import foo; void bar(int id) { baz!((a) { id = 0; // passing a delegate as template argument does not allow // accessing its environment })(); } dmd foo.d bar.d results in: bar.d(5): Error: function bar.bar compiler error, parameter 'id', bugzilla 2962? dmd: glue.c:717: virtual void FuncDeclaration::toObjFile(int): Assertion `0' failed. Unfortunately I couldn't reduce the test case any further. If the delegate has no parameter I cannot reproduce the compiler assertion failure. And of course if id is not accessed from within the delegate the problem goes away. This is at the core of the problem. As pointed out in the comments the specified order of files is important. I.e. dmd bar.d foo.d compiles just fine. I used DMD64 D Compiler v2.060. I'm reopening this bug.

Using a local variable inside the delegate instead of a parameter of the calling function results in incorrect code: ---------------- module b; auto call(alias fun)() { return fun(0); } ---------------- module a; import b; import std.stdio; void main() { int x = 1; writeln(call!(a => x)); // should print the value of x } ---------------- $ dmd b.d a.d -ofa && ./a -771364480 (Using DMD64 D Compiler v2.060 on Linux)

Comments 42 and 43 refer to an entirely different bug. The test cases in those comments were fixed by today's fix to bug 6395.

Commit pushed to master at https://github.com/D-Programming-Language/dmd https://github.com/D-Programming-Language/dmd/commit/ba0b1d73fff98be1440d972fa88bb730d136b10c Add fail_compilation test for bug 2962