Bug 5248 – CTFE Segfault when calling a function on an enum struct

This code makes the DMD compiler segfault struct LeafType { string Compile_not_ovloaded() { return "expression"; } }; struct MatrixASTNode { LeafType Right; string Compile() { return Right.Compile_not_ovloaded(); } }; void main() { enum AST = MatrixASTNode(); enum s=AST.Compile(); } Inspecting dmd with a debugger suggests this is caused by a stack overflow. These two functions in interpret.c call each other repeatedly. Expression *ThisExp::interpret(InterState *istate) { if (istate && istate->localThis) return istate->localThis->interpret(istate); error("value of 'this' is not known at compile time"); return EXP_CANT_INTERPRET; } Expression *DotVarExp::interpret(InterState *istate) { Expression *e = EXP_CANT_INTERPRET; #if LOG printf("DotVarExp::interpret() %s\n", toChars()); #endif Expression *ex = e1->interpret(istate); // <- we never get past here /* rest of function */ } If you turn logging on for the file you get this: CallExp::interpret() MatrixASTNode(LeafType()).Compile() ******** FuncDeclaration::interpret(istate = (nil)) Compile cantInterpret = 0, semanticRun = 5 StructLiteralExp::interpret() MatrixASTNode(LeafType()) StructLiteralExp::interpret() LeafType() CompoundStatement::interpret() ExpStatement::interpret(assert(&this,"null this")) AssertExp::interpret() assert(&this,"null this") StructLiteralExp::interpret() MatrixASTNode(LeafType()) StructLiteralExp::interpret() LeafType() ReturnStatement::interpret(this.Right.Compile_not_ovloaded()) CallExp::interpret() this.Right.Compile_not_ovloaded() ******** FuncDeclaration::interpret(istate = 0xbfe685a0) Compile_not_ovloaded cantInterpret = 0, semanticRun = 5 DotVarExp::interpret() this.Right StructLiteralExp::interpret() MatrixASTNode(LeafType()) StructLiteralExp::interpret() LeafType() CompoundStatement::interpret() ExpStatement::interpret(assert(&this,"null this")) AssertExp::interpret() assert(&this,"null this") DotVarExp::interpret() this.Right DotVarExp::interpret() this.Right DotVarExp::interpret() this.Right DotVarExp::interpret() this.Right ...an so on until stack overflow The reason for the recursion happens is that in the context of DotVarExp::interpret istate->localThis == this so in ThisExp::interpret the statement istate->localThis->interpret(istate); goes right back to DotVarExp::interpret again.

Slight simplification, you don't need a main to cause this: struct LeafType { string Compile_not_ovloaded() { return "expression"; } }; struct MatrixASTNode { LeafType Right; string Compile() { return Right.Compile_not_ovloaded(); } }; enum AST = MatrixASTNode(); enum s=AST.Compile(); Makes the backtrace shorter.

Okay, had time for a little more poking. Execution moves to DotVarExp::interpret and the endless loop from here: Expression *AssertExp::interpret(InterState *istate) { Expression *e; Expression *e1; if( this->e1->op == TOKaddress) { // Special case: deal with compiler-inserted assert(&this, "null this") AddrExp *ade = (AddrExp *)this->e1; if (ade->e1->op == TOKthis && istate->localThis) if (ade->e1->op == TOKdotvar // <--- something is fishy here && ((DotVarExp *)(istate->localThis))->e1->op == TOKthis) return getVarExp(loc, istate, ((DotVarExp*)(istate->localThis))->var); else return istate->localThis->interpret(istate); } Apparently the compiler puts in an implicit assert(&this,"this is null") at the beginning of member functions. Now ade->e1->op == TOKthis is true by the time execution reaches the inner if so why is ade->e1->op == TOKdotvar being tested? That statement is always false. If we replace ade->e1->op == TOKdotvar with true the compiler accepts the sample program. The question is, what is getVarExp? and why might return istate->localThis->interpret(istate); be the wrong thing to do?

Excellent! (In reply to comment #2) > Okay, had time for a little more poking. Execution moves to > DotVarExp::interpret and the endless loop from here: > > > Expression *AssertExp::interpret(InterState *istate) > { Expression *e; > Expression *e1; > > if( this->e1->op == TOKaddress) > { // Special case: deal with compiler-inserted assert(&this, "null this") > AddrExp *ade = (AddrExp *)this->e1; > if (ade->e1->op == TOKthis && istate->localThis) > if (ade->e1->op == TOKdotvar // <--- something is fishy here > && ((DotVarExp *)(istate->localThis))->e1->op == TOKthis) > return getVarExp(loc, istate, > ((DotVarExp*)(istate->localThis))->var); > else > return istate->localThis->interpret(istate); > } > > Apparently the compiler puts in an implicit assert(&this,"this is null") at the > beginning of member functions. > > Now ade->e1->op == TOKthis is true by the time execution reaches the inner if > so why is ade->e1->op == TOKdotvar being tested? That statement is always > false. If we replace ade->e1->op == TOKdotvar with true the compiler accepts > the sample program. > > The question is, what is getVarExp? and why might return > istate->localThis->interpret(istate); be the wrong thing to do? If it is returning by reference, it needs to return a VarExp, whereas localThis->interpret() returns the value of 'this' (ie, an rvalue). You're right about the problem line. That should definitely be: - if (ade->e1->op == TOKdotvar + if (istate->localThis->op == TOKdotvar && ((DotVarExp *)(istate->localThis))->e1->op == TOKthis) return getVarExp(loc, istate,

Okay, I made that change and it works for me now.

Fixed https://github.com/D-Programming-Language/dmd/commit/07385f16bf24724e8f809274ec560d2c9c5e65f9