Bug 3560 – foreach over nested function generates wrong code

import std.stream: File; import std.stdio: writefln; void outer() { auto auth_file = new File("path-to-existing-file"); writefln("outer: 0x%X", &auth_file); int inner(int delegate(ref ubyte[]) dg) { writefln("inner: 0x%X", &auth_file); return 0; } foreach (entry; &inner) { //... } } void main() { outer(); } -------------------- expected output: outer: 0xB7D19E44 inner: 0xB7D19E44 actual output: outer: 0xB7D19E44 inner: 0x4

Reduced test case (this example segfaults at runtime) shows it is related to varargs. ------------------------ void bug3560(int ...){ } void main() { int localvar = 7; bug3560(2); // this call must be made before the foreach int inner(int delegate(ref int) dg) { int k = localvar; return 0; } foreach (entry; &inner) { } }

Further reduction shows that it doesn't require varargs. Raising priority to critical, since it means that virtually all closures are broken. What's happening: on the call to inner, EAX should hold a pointer to the closure variables (on the heap). But when there's another function call, it's using EBX instead, and the EAX value is whatever garbage is left after the function call. So it gets the calling convention wrong. ==================== void bug3560(int x){ } void main() { int localvar = 7; int inner(int delegate(ref int) dg) { int k = localvar; // BUG: &localvar == 0x24 (!) return 0; } bug3560(0x20); foreach (entry; &inner) { } }

It doesn't even need a function call, and doesn't need a closure. Anything which modifies the EAX register before the foreach will do it. This test case generates bad code (runtime segfault) even on D1 (even old versions, eg D1.020). The inner function assumes that EAX contains the context pointer, but the foreach code doesn't set EAX. Instead, it's passing the context pointer in EBX. ===================== void main() { int localvar = 7; int inner(int delegate(ref int) dg) { int k = localvar; return 0; } int a = localvar * localvar; // This modifies the EAX register foreach (entry; &inner) { } }

This is a front-end problem. ForeachStatement::semantic() immediately runs aggr->semantic(). In this case, aggr is an address of a nested function 'inner'. AddrExp->semantic() turns it into a DelegateExp. Later on in Foreach::semantic, if the aggregate is of delegate type, it wraps it in a CallExp, then runs CallExp::semantic. The problem is that CallExp assumes that semantic has NOT been run on its argument. This works fine if the aggregate is a delegate variable. But if it's a delegate expression, CallExp transforms the DelegateExp for 'inner' into a DotVarExp(inner, inner). And then it's a mess. CallExp needs to be passed 'inner', not the delegateExp. I can see two possible fixes. (1) Change CallExp so that changes DelegateExp(f,f) into CallExp(f), if f is a nested function, instead of changing it into a DotVarExp. OR (2) If it's a delegate expression for a nested function, call the function directly. This patch implements the second method. statement.c, Foreach::semantic(), line 1891 else if (tab->ty == Tdelegate) { /* Call: * aggr(flde) */ Expressions *exps = new Expressions(); exps->push(flde); + if (aggr->op == TOKdelegate && + ((DelegateExp *)aggr)->func->isNested()) + e = new CallExp(loc, ((DelegateExp *)aggr)->e1, exps); + else + e = new CallExp(loc, aggr, exps); - e = new CallExp(loc, aggr, exps); e = e->semantic(sc);