Bug 23676 – Static foreach hangs compilation for some time

The main, (mostly) reduced code is here: https://gist.github.com/BradleyChatha/873077996b36c9f575fa2410e03705d5 DMD version: DMD64 D Compiler v2.101.2 LDC2 version (has same issue): LDC - the LLVM D compiler (1.30.0): based on DMD v2.100.1 and LLVM 14.0.6 built with LDC - the LLVM D compiler (1.28.1) Default target: arm64-apple-darwin21.6.0 Host CPU: cyclone http://dlang.org - http://wiki.dlang.org/LDC Main issue: If you don't wrap the body of the `static if` on Line 70 inside curly braces, the compiler begins to error out with the following: ``` to_report.d(85): Error: `@safe` function `lexer.Lexer.nextVaryingLengthToken!(safePeek, Operator).nextVaryingLengthToken` cannot call `@system` function `lexer.Lexer.nextVaryingLengthToken!(safePeek, Operator).nextVaryingLengthToken.tryLexLongerOperators!(noreturn).tryLexLongerOperators` to_report.d(58): `lexer.Lexer.nextVaryingLengthToken!(safePeek, Operator).nextVaryingLengthToken.tryLexLongerOperators!(noreturn).tryLexLongerOperators` is declared here ``` It then proceeds to hang and use max out its CPU usage for a few minutes, before exiting. However, if you wrap the static if's body in braces, it will succesfully compile without the weird performance issues. Side issue: I'm very uncertain about the cause of the hang; if you delete any field within the `Token.Type` enum, the hang reduces to a couple of seconds, instead of minutes.

This is a serious issue. Bumping severity to "major"

Even more strangely: If you delete any line within the body of the case statement that is guarded by the fault `static if`, compilation succeeds. What a very strange edge case this is hitting.

Marking the nested function as @safe prevents the error message; allows the compilation to succeed, however the compiler does still hang for a while before it finishes.

I manually reduced it to get rid of phobos and other unrelated features: module lexer; enum Type { operatorSlash, operatorSlashEqual, operatorAnd, operatorAndEqual, operatorAndAnd, operatorQuestion, operatorDollar, operatorEqual, operatorEqualEqual, operatorStar, operatorStartEqual, } struct Lexer { size_t _index; @safe: void nextOperator() { const couldLex = nextVaryingLengthToken(); } // Complex reading // bool nextVaryingLengthToken() { alias TokenTypes = __traits(allMembers, Type); bool tryLexLongerOperators(alias TokenType)() { static foreach(type; TokenTypes) { _index++; if(tryLexLongerOperators!type) return true; return true; } } return tryLexLongerOperators!noreturn; } } My hunch is that `static foreach` ends up generating too many recursive calls to tryLexLongerOperators, but I don't know exactly what causes the hang.

The `static foreach` is not great (btw you can drop `static`) but the real culprit is the access to the struct member `_index` from the nested functions. That would be a problem of counter performance caused by the creation of closures. kcachegrind shows that the costy thing would be a cycle between `FuncDeclaration.needsClosure` and checkEscapingSiblings.

Yeah, it does appear to be exponential. If you change the static foreach to include a loop index; and then guard its body under a `static if(i < 1)`, the slowdown appears to start at i < 8. So in other words, my code is bad and I should feel bad ;) (or more seriously it could be a combination of bad code and a bad pathway in the frontend). Curiously, `i < 1` allows the function to compile perfectly fine however `i < 2` makes you run into the `@safe function can't call @system function` error. Is that a seperate attribute inference bug or am I missing something, because you can easily add @safe onto the nested function to solve that problem.

Though that also still doesn't explain the strange behaviour where using a bracefull body works fine, without causing the hang or the inference error, and using a braceless body causes the errors and the hang.

Reduced a bit further: ``` void f() { int i; bool g(int I)() { static foreach(j; 0..11) { i++; return g!j(); } } g!0; } ``` The 11 is the number of enum members of Type. The amount of calls to `checkEscapingSiblings` grows exponentially based on the loop length, which is why compilation hangs.

@dkorpel created dlang/dmd pull request #14886 "Fix 23676 - Static foreach hangs compilation for some time" fixing this issue: - Fix 23676 - Static foreach hangs compilation for some time https://github.com/dlang/dmd/pull/14886

dlang/dmd pull request #14886 "Fix 23676 - Static foreach hangs compilation for some time" was merged into master: - f0b5ce49c72caa6c9393195b46adf32dd503cd92 by Dennis Korpel: Fix 23676 - Static foreach hangs compilation for some time https://github.com/dlang/dmd/pull/14886