Bug 22114 – case of ICE happening during CTFE

When attempting to build a complex software with various data structures, I came across a parser error on all DLang compilers: DMD, LDC2, and RDMD. All of them have the same problem coming from DMD implementation for Token parsing particularly on line 914. I've narrowed down the code to the following reproducible code here: ```d module testproblem; public class Test1(T) { private Test2!T val; this() { val = new Test2!T(this); } private class Test2(T) { private Test1!(T) m_source; this(Test1!T source) { m_source = source; } } } public class Demo { auto val = new Test1!int(); } ``` The error shown is as followed after running the command: dmd -c -wi -Isource/ -oftest.o source/testproblem.d -v //////////////////////////////////////////////////////// predefs DigitalMars LittleEndian D_Version2 all D_SIMD Posix ELFv1 linux CRuntime_Glibc CppRuntime_Gcc D_InlineAsm_X86_64 X86_64 D_LP64 D_PIC assert D_ModuleInfo D_Exceptions D_TypeInfo D_HardFloat binary dmd version v2.097.0 config /etc/dmd.conf DFLAGS -I/usr/include/dlang/dmd -L-L/usr/lib -L--export-dynamic -fPIC parse testproblem importall testproblem import object (/usr/include/dlang/dmd/object.d) import core.attribute (/usr/include/dlang/dmd/core/attribute.d) import core.internal.hash (/usr/include/dlang/dmd/core/internal/hash.d) import core.internal.traits (/usr/include/dlang/dmd/core/internal/traits.d) import core.internal.entrypoint (/usr/include/dlang/dmd/core/internal/entrypoint.d) import core.internal.array.appending (/usr/include/dlang/dmd/core/internal/array/appending.d) import core.internal.array.comparison (/usr/include/dlang/dmd/core/internal/array/comparison.d) import core.internal.array.equality (/usr/include/dlang/dmd/core/internal/array/equality.d) import core.internal.array.casting (/usr/include/dlang/dmd/core/internal/array/casting.d) import core.internal.array.concatenation (/usr/include/dlang/dmd/core/internal/array/concatenation.d) import core.internal.array.construction (/usr/include/dlang/dmd/core/internal/array/construction.d) import core.internal.array.capacity (/usr/include/dlang/dmd/core/internal/array/capacity.d) import core.internal.dassert (/usr/include/dlang/dmd/core/internal/dassert.d) import core.atomic (/usr/include/dlang/dmd/core/atomic.d) import core.internal.attributes (/usr/include/dlang/dmd/core/internal/attributes.d) import core.internal.atomic (/usr/include/dlang/dmd/core/internal/atomic.d) import core.internal.destruction (/usr/include/dlang/dmd/core/internal/destruction.d) import core.internal.moving (/usr/include/dlang/dmd/core/internal/moving.d) import core.internal.postblit (/usr/include/dlang/dmd/core/internal/postblit.d) import core.internal.switch_ (/usr/include/dlang/dmd/core/internal/switch_.d) import core.builtins (/usr/include/dlang/dmd/core/builtins.d) semantic testproblem semantic2 testproblem semantic3 testproblem code testproblem Segmentation fault (core dumped)

In reference to Token Error, I am referring to this: https://github.com/dlang/dmd/blob/c436df7580be08481386e13da41c08e91f46b07f/src/dmd/tokens.d#L914

Not a parser error, the trace output suggests an error in the backend. But the code fails with current master in the CTFE engine: [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x0000000008377354 in _D3dmd10dinterpret16scrubReturnValueFKxSQBm7globals3LocCQCc10expression10ExpressionZ7scrubSEMFCQDpQBn16StructLiteralExpZQCn (__capture=0x7ffffffecc50, sle=0x84f2d9b <Visitor::visit(ClassReferenceExp*)+31>) at src/dmd/dinterpret.d:6548 6548 sle.ownedByCtfe = OwnedBy.code; #0 0x0000000008377354 in _D3dmd10dinterpret16scrubReturnValueFKxSQBm7globals3LocCQCc10expression10ExpressionZ7scrubSEMFCQDpQBn16StructLiteralExpZQCn (__capture=0x7ffffffecc50, sle=0x84f2d9b <Visitor::visit(ClassReferenceExp*)+31>) at src/dmd/dinterpret.d:6548 #1 0x0000000008376fa8 in _D3dmd10dinterpret16scrubReturnValueFKxSQBm7globals3LocCQCc10expression10ExpressionZQBd (e=0x7ffffffece70, loc=...) at src/dmd/dinterpret.d:6563 #2 0x00000000083772ff in _D3dmd10dinterpret16scrubReturnValueFKxSQBm7globals3LocCQCc10expression10ExpressionZ10scrubArrayMFPSQDu4root5array__T5ArrayTQCrZQlbZQCz (__capture=0x7ffffffecda0, structlit=true, elems=0x7ffffdf664e0) at src/dmd/dinterpret.d:6538 #3 0x0000000008377386 in _D3dmd10dinterpret16scrubReturnValueFKxSQBm7globals3LocCQCc10expression10ExpressionZ7scrubSEMFCQDpQBn16StructLiteralExpZQCn (__capture=0x7ffffffecda0, sle=0x7ffffdf66650) at src/dmd/dinterpret.d:6553 #4 0x0000000008376fa8 in _D3dmd10dinterpret16scrubReturnValueFKxSQBm7globals3LocCQCc10expression10ExpressionZQBd (e=0x7ffffdf666b0, loc=...) at src/dmd/dinterpret.d:6563 #5 0x00000000083772ff in _D3dmd10dinterpret16scrubReturnValueFKxSQBm7globals3LocCQCc10expression10ExpressionZ10scrubArrayMFPSQDu4root5array__T5ArrayTQCrZQlbZQCz (__capture=0x7ffffffecef0, structlit=true, elems=0x7ffffdf66460) at src/dmd/dinterpret.d:6538 #6 0x0000000008377386 in _D3dmd10dinterpret16scrubReturnValueFKxSQBm7globals3LocCQCc10expression10ExpressionZ7scrubSEMFCQDpQBn16StructLiteralExpZQCn (__capture=0x7ffffffecef0, sle=0x7ffffdf666e0) at src/dmd/dinterpret.d:6553 #7 0x0000000008376fa8 in _D3dmd10dinterpret16scrubReturnValueFKxSQBm7globals3LocCQCc10expression10ExpressionZQBd (e=0x7ffffdf665c0, loc=...) at src/dmd/dinterpret.d:6563 #8 0x0000000008363872 in _D3dmd10dinterpret13ctfeInterpretFCQBh10expression10ExpressionZQBd (e=0x7ffffdf665c0) at src/dmd/dinterpret.d:102 #9 0x000000000840efc1 in Expression::ctfeInterpret() (this=0x7ffffdf665c0) at src/dmd/expression.d:1598 #10 0x0000000008464fd1 in _D3dmd7initsem19initializerSemanticRCQBj4init11InitializerPSQCg6dscope5ScopeKCQCy5mtype4TypeEQDnQCe13NeedInterpretZ8visitExpMFCQEvQDm14ExpInitializerZQEk (__capture=0x7ffffffed0f0, i=0x7fffff6c7f10) at src/dmd/initsem.d:571 #11 0x0000000008462fa6 in initializerSemantic(Initializer*, Scope*, Type*&, NeedInterpret) (init=0x7fffff6c7f10, sc=0x7ffffdf61e60, tx=@0x7fffff6c7fc0: 0x7ffffdf5d100, needInterpret=<incomplete type>) at src/dmd/initsem.d:811 #12 0x00000000084b50f2 in Semantic2Visitor::visit(VarDeclaration*) (this=0x7ffffffed190, vd=0x7fffff6c7f40) at src/dmd/semantic2.d:259 #13 0x000000000835fcc2 in VarDeclaration::accept(Visitor*) (this=0x7fffff6c7f40, v=0x7ffffffed190) at src/dmd/declaration.d:1610 #14 0x00000000084b495e in semantic2(Dsymbol*, Scope*) (dsym=0x7fffff6c7f40, sc=0x7ffffdf61e60) at src/dmd/semantic2.d:79 #15 0x00000000084b5e7d in Semantic2Visitor::visit(AggregateDeclaration*) (this=0x7ffffffed270, ad=0x7fffff6c8080) at src/dmd/semantic2.d:612 #16 0x00000000084b5ef1 in Semantic2Visitor::visit(ClassDeclaration*) (this=0x7ffffffed270, cd=0x7fffff6c8080) at src/dmd/semantic2.d:669 #17 0x000000000835bf5e in ClassDeclaration::accept(Visitor*) (this=0x7fffff6c8080, v=0x7ffffffed270) at src/dmd/dclass.d:995 #18 0x00000000084b495e in semantic2(Dsymbol*, Scope*) (dsym=0x7fffff6c8080, sc=0x7ffffdf61d30) at src/dmd/semantic2.d:79 #19 0x00000000084b5b7f in Semantic2Visitor::visit(AttribDeclaration*) (this=0x7ffffffed340, ad=0x7fffff6c83d0) at src/dmd/semantic2.d:522 #20 0x00000000084b1abf in ParseTimeVisitor<ASTCodegen>::visit(VisibilityDeclaration*) (this=0x7ffffffed340, s=0x7fffff6c83d0) at src/dmd/parsetimevisitor.d:76 #21 0x000000000832489a in VisibilityDeclaration::accept(Visitor*) (this=0x7fffff6c83d0, v=0x7ffffffed340) at src/dmd/attrib.d:685 #22 0x00000000084b495e in semantic2(Dsymbol*, Scope*) (dsym=0x7fffff6c83d0, sc=0x7ffffdf5f990) at src/dmd/semantic2.d:79 #23 0x00000000084b5486 in Semantic2Visitor::visit(Module*) (this=0x7ffffffed3f0, mod=0x7fffff6c50d0) at src/dmd/semantic2.d:337 #24 0x000000000838217a in Module::accept(Visitor*) (this=0x7fffff6c50d0, v=0x7ffffffed3f0) at src/dmd/dmodule.d:1527 #25 0x00000000084b495e in semantic2(Dsymbol*, Scope*) (dsym=0x7fffff6c50d0, sc=0x0) at src/dmd/semantic2.d:79 #26 0x000000000830caf8 in _D3dmd4mars7tryMainFmPPxaKSQz7globals5ParamZi (params=..., argv=0x7ffffffeddb8, argc=4) at src/dmd/mars.d:495 #27 0x000000000830eb4a in D main (_param_0=...) at src/dmd/mars.d:1070

(In reply to moonlightsentinel from comment #2) > Not a parser error, the trace output suggests an error in the backend. > > But the code fails with current master in the CTFE engine: Alright, prior to reducing the code to minimal reproducible code, I got this error which led me to thinking to a conclusion that it's a parser error: core.exception.RangeError@src/dmd/tokens.d(914): Range violation

(In reply to Tyler from comment #3) > Alright, prior to reducing the code to minimal reproducible code, I got this > error which led me to thinking to a conclusion that it's a parser error: > > core.exception.RangeError@src/dmd/tokens.d(914): Range violation Token.toString is a utility method not only used by the parser. But it sounds like your reduction was not correct and found a different error instead.

(In reply to moonlightsentinel from comment #4) > (In reply to Tyler from comment #3) > > Alright, prior to reducing the code to minimal reproducible code, I got this > > error which led me to thinking to a conclusion that it's a parser error: > > > > core.exception.RangeError@src/dmd/tokens.d(914): Range violation > > Token.toString is a utility method not only used by the parser. But it > sounds like your reduction was not correct and found a different error > instead. Alright, I'll try to investigate further on it and report the bug separately for the token error. Thank you for checking into it.

Sorry for long delay, I've produced another minimal reproducible code, but it seems to be very closely relevant to above reproducible code and yet it printed different stack trace error. I've compiled dmd from master branch for the following snippet: ```d module test; public @nogc class TestA(T) { private TestB!T valA; private TestB!T valB; this() { valB = valA = new TestB!T(this); } private @nogc class TestB(T) { private TestA!(T) m_source; this(TestA!T source) { m_source = source; } } } public class Demo { auto val = new TestA!int(); } ``` And it generated the following stack trace: (lldb) run -c -I./source/ -oftest.o ./source/test.d -g Process 23902 launched: '/home/tyler/Repos/dlang/dmd/generated/linux/release/64/dmd' (x86_64) --- ERROR: This is a compiler bug. Please report it via https://issues.dlang.org/enter_bug.cgi with, preferably, a reduced, reproducible example and the information below. DustMite (https://github.com/CyberShadow/DustMite/wiki) can help with the reduction. --- DMD v2.097.0-352-gc654f1b73 predefs DigitalMars LittleEndian D_Version2 all D_SIMD Posix ELFv1 linux CRuntime_Glibc CppRuntime_Gcc D_InlineAsm_X86_64 X86_64 D_LP64 D_PIC assert D_ModuleInfo D_Exceptions D_TypeInfo D_HardFloat binary /home/tyler/Repos/dlang/dmd/generated/linux/release/64/dmd version v2.097.0-352-gc654f1b73 config /home/tyler/Repos/dlang/dmd/generated/linux/release/64/dmd.conf DFLAGS -I/home/tyler/Repos/dlang/dmd/generated/linux/release/64/../../../../../druntime/import -I/home/tyler/Repos/dlang/dmd/generated/linux/release/64/../../../../../phobos -L-L/home/tyler/Repos/dlang/dmd/generated/linux/release/64/../../../../../phobos/generated/linux/release/64 -L--export-dynamic -fPIC --- core.exception.RangeError@src/dmd/tokens.d(1019): Range violation ---------------- ??:? _d_arrayboundsp [0x555555bac239] src/dmd/tokens.d:1019 pure nothrow @nogc @safe immutable(char)[] dmd.tokens.Token.toString(uint) [0x555555ace774] src/dmd/tokens.d:1014 _ZN5Token7toCharsEj [0x555555ace73a] src/dmd/dinterpret.d:6825 dmd.expression.Expression dmd.dinterpret.copyRegionExp(dmd.expression.Expression) [0x5555558ed7de] src/dmd/dinterpret.d:6720 dmd.expression.Expression dmd.dinterpret.copyRegionExp(dmd.expression.Expression) [0x5555558ed515] src/dmd/dinterpret.d:6700 void dmd.dinterpret.copyRegionExp(dmd.expression.Expression).copyArray(dmd.root.array.Array!(dmd.expression.Expression).Array*) [0x5555558ed8b0] src/dmd/dinterpret.d:6710 void dmd.dinterpret.copyRegionExp(dmd.expression.Expression).copySE(dmd.expression.StructLiteralExp) [0x5555558ed8ef] src/dmd/dinterpret.d:6738 dmd.expression.Expression dmd.dinterpret.copyRegionExp(dmd.expression.Expression) [0x5555558ed57d] src/dmd/dinterpret.d:6720 dmd.expression.Expression dmd.dinterpret.copyRegionExp(dmd.expression.Expression) [0x5555558ed515] src/dmd/dinterpret.d:6700 void dmd.dinterpret.copyRegionExp(dmd.expression.Expression).copyArray(dmd.root.array.Array!(dmd.expression.Expression).Array*) [0x5555558ed8b0] src/dmd/dinterpret.d:6710 void dmd.dinterpret.copyRegionExp(dmd.expression.Expression).copySE(dmd.expression.StructLiteralExp) [0x5555558ed8ef] src/dmd/dinterpret.d:6738 dmd.expression.Expression dmd.dinterpret.copyRegionExp(dmd.expression.Expression) [0x5555558ed57d] src/dmd/dinterpret.d:6720 dmd.expression.Expression dmd.dinterpret.copyRegionExp(dmd.expression.Expression) [0x5555558ed515] src/dmd/dinterpret.d:99 dmd.expression.Expression dmd.dinterpret.ctfeInterpret(dmd.expression.Expression) [0x5555558d960b] src/dmd/expression.d:1598 _ZN10Expression13ctfeInterpretEv [0x555555984c18] src/dmd/initsem.d:571 dmd.init.Initializer dmd.initsem.initializerSemantic(dmd.init.Initializer, dmd.dscope.Scope*, ref dmd.mtype.Type, dmd.init.NeedInterpret).visitExp(dmd.init.ExpInitializer) [0x5555559da75c] src/dmd/initsem.d:811 _Z19initializerSemanticP11InitializerP5ScopeRP4Type13NeedInterpret [0x5555559d8731] src/dmd/semantic2.d:259 _ZN16Semantic2Visitor5visitEP14VarDeclaration [0x555555a2a699] src/dmd/declaration.d:1610 _ZN14VarDeclaration6acceptEP7Visitor [0x5555558d60b1] src/dmd/semantic2.d:79 _Z9semantic2P7DsymbolP5Scope [0x555555a29f05] src/dmd/semantic2.d:612 _ZN16Semantic2Visitor5visitEP20AggregateDeclaration [0x555555a2b424] src/dmd/semantic2.d:669 _ZN16Semantic2Visitor5visitEP16ClassDeclaration [0x555555a2b498] src/dmd/dclass.d:995 _ZN16ClassDeclaration6acceptEP7Visitor [0x5555558d23e1] src/dmd/semantic2.d:79 _Z9semantic2P7DsymbolP5Scope [0x555555a29f05] src/dmd/semantic2.d:522 _ZN16Semantic2Visitor5visitEP17AttribDeclaration [0x555555a2b126] src/dmd/parsetimevisitor.d:76 _ZN16ParseTimeVisitorI10ASTCodegenE5visitEP21VisibilityDeclaration [0x555555a27066] src/dmd/attrib.d:685 _ZN21VisibilityDeclaration6acceptEP7Visitor [0x55555589ad99] src/dmd/semantic2.d:79 _Z9semantic2P7DsymbolP5Scope [0x555555a29f05] src/dmd/semantic2.d:337 _ZN16Semantic2Visitor5visitEP6Module [0x555555a2aa2d] src/dmd/dmodule.d:1527 _ZN6Module6acceptEP7Visitor [0x5555558f7e35] src/dmd/semantic2.d:79 _Z9semantic2P7DsymbolP5Scope [0x555555a29f05] src/dmd/mars.d:495 int dmd.mars.tryMain(ulong, const(char)**, ref dmd.globals.Param) [0x55555588324b] src/dmd/mars.d:1074 _Dmain [0x555555885355] Process 23902 exited with status = 1 (0x00000001)

For some reasons, it seems to be looking for an index of 18668 at tokens.d:1019 for tochars string array constant. Relevant frame information when debugging: ``` * frame #0: 0x0000555555ace74f dmd`_D3dmd6tokens5Token8toStringFNaNbNiNfkZAya(value=18668) at tokens.d:1019 frame #1: 0x0000555555ace73b dmd`Token::toChars(value=18668) at tokens.d:1014 ```

As it turns out, the problem is that the value given for aforementioned tochars index is coming from a de-referenced object that is pointing to an invalid address, because I have checked conditional breakpoint to check constructor for Expression abstract class specifically in dmd/src/dmd/expression.d at line 663 which never have op set to 18668. I am pretty unfamiliar with the implementation of DMD, so I would appreciate any tips or suggestions here to help address this blocking issue.

Created attachment 1823 View of KDBG for Invalid Address Derefencing in DMD Compiler Attached an image to show what I'm seeing here in KDBG for debugging this issue.

Gave up with DLang due to the lack of interest in resolving this issue. I'm closing this issue and moving on to other programming language.

leaving the D scene does not make this problem invalid so dont close, it's just that contributors have no obligations, they can peek and fix whatever they want.

I just checked this was resolved by https://github.com/dlang/dmd/pull/13109 I'll push these tests for good measure. *** This issue has been marked as a duplicate of issue 22292 ***