Bug 18698 – static foreach + __traits(allMembers, moduleName)

I just tried to write some code that I've been using for 10 years... but it's not working today. This works: module test; pragma(msg, __traits(allMembers, test)); Prints: tuple("object", "std", "numberOf", "arr", "t") This should work, but doesn't: module test; static foreach(m; __traits(allMembers, test)) { pragma(msg, m); } Prints: Compiling C:\Code\blah.d failed! Nice informative error ;) I've been scanning the module for things since I started using D, but today, it doesn't seem to work... This is actually really serious. A huge amount of my compile-time machinery is broken now! I can't serialise, or do any of my DLL machinery, dynamic language bindings... :/

I try this: module test; alias Members = __traits(allMembers, test); > Error: basic type expected, not `__traits` We *really* need to fix that one... So: module test; import std.meta; alias Members = AliasSeq!(__traits(allMembers, test)); pragma(msg, Members); > tuple("object", "std", "arr", "t", "Members") Okay, that's good. static foreach(m; Members) { pragma(msg, m); } Error: template instance AliasSeq!(__traits(allMembers, test)) recursive template expansion Oh dear...

lol. another `static foreach` great feature: `allMembers` tries to expand `static foreach`, effectively executing it's body, whith calls `allMembers`, which tries to expand `static foreach`, effectively...

I barely understood a word of that post. Point is, this used to work... I've been using this for almost 10 years to do a bunch of codegen for stuff in module scope.

Static foreach has been added in summer 2017 - how could you have been using this for ten years? (if this really is a regression, please describe it better so that everyone understands it. Otherwise I think ketmar's assessment of this being an oversight when static foreach was added.

[i]>I've been using this for almost 10 years[/i] static foreach? so you really have a time machine, or a specially-crafted compiler version you never gave others?! please, can you show us the 10 yo code with `static foreach`, and the 10 yo old compiler you used to compile it? this is normal thing for new language feature: some cases are missed. it will eventually be fixed, and in the meantime you can write the code like us, mere mortals did for those 10 years: without `static foreach`.

I used 'foreach', which doesn't work anymore. Since static foreach was added, the non-static one doesn't work the same. static foreach should to all the static stuff that foreach used to do.

`foreach` never worked at the top level. and non-static forach works *exactly* the same as before. you are clearly has a very different compiler than the rest of us.

Non-static foreach doesn't seem to be the same. There are new errors with old usage of foreach instructing to use static foreach instead. And it definitely worked at top level. I was using it to generate top level binding stub functions almost as long as I've been using D. It would be inside a mixin template mixed in at top level, that's all.

>Non-static foreach doesn't seem to be the same. example, please. *nothing* was changed in old foreach. >And it definitely worked at top level. example, please. i have several different compiler versions at hand, starting from 2.073 (WAY before static foreach), and top-level foreach doesn't work in all of them. please, give example code and compiler version where it worked. thank you.

Remove the 'static' from my OP and you have it. It definitely worked. They flew me to dconf in 2013 and I gave a whole lecture about it. I don't work there anymore, so I don't have the code. I was starting to write a new version. Emitting an error instructing me to insert 'static' is at least once change in old foreach.

module test; foreach(m; __traits(allMembers, test)) { pragma(msg, m); } rdmd --force --compiler=ldc --eval="pragma(msg, __VERSION__);" 2073L rdmd --force --compiler=ldc -c test.d test.d(2): Error: declaration expected, not 'foreach' test.d(2): Error: declaration expected, not '__traits' test.d(5): Error: unrecognized declaration 2.073 is *way* before `static foreach`. so please, code sample and compiler version. if it used to work, and now doesn't, this is clearly a regression nobody noticed. the same for old `foreach` code that doesn't work as it used to work before, please.

I'd love to paste code... But I wrote it in 2012 at a company I don't work at anymore. Think D2.042... I'll see what I can do when I'm not in bed on a phone. But I have no idea what now I can offer. I used to take function prototypes at top level, and mixin their bodies at the same scope. My entire system was based on this. They didn't fly me to dconf to talk about code that didn't compile.

>Think D2.042 easy deal: http://downloads.dlang.org/releases/2010/ [pts/44:ketmar]:D/_old_dmd% ./dmd2/linux/bin/dmd Digital Mars D Compiler v2.042 Copyright (c) 1999-2010 by Digital Mars written by Walter Bright [pts/44:ketmar]:D/_old_dmd% ./dmd2/linux/bin/dmd -c test.d test.d(2): Declaration expected, not 'foreach' test.d(2): Declaration expected, not '__traits' test.d(5): unrecognized declaration exactly the same error with 2.039. should i go even further back in time? or, fast-forward to 2.059: still the same thing. it. never. worked.

Here's the regression tester with all 19 versions from 2.060 to 2.079: https://run.dlang.io/is/ZBosCr Anyhow, I think we all agree that Manu's use case should work, so let's focus on fixing that (and avoid the off-topic discussion whether this may or may nor have worked a long time ago).

Okay, I must have had some workaround... I don't remember what exactly, it was some years ago. I probably wrapped it in layers until it worked. Regardless, it should work. My colleague just tried to do this and asked me why it didn't work... I have no recollection of workarounds I might have used.

I remember my work-around; I ran the foreach inside a static module constructor! From there I generated my reflection information about the module and registered it at runtime (in the module constructor), rather than emit data directly into the module scope. In this instance, I really need to emit functions into the module scope, and not do runtime registration like I did last time. It would be really really nice to fix this issue!

As a workaround for now, you can use a string mixin: import std.conv : to; string create() { string s = ""; static foreach (i, e; __traits(allMembers, mixin(__MODULE__))) { s ~= "int n"~i.to!string~";"; } return s; } mixin(create()); pragma(msg, __traits(allMembers, mixin(__MODULE__))); // tuple("object", "create", "n0", "n1") The root cause is, as Ketmar pointed out in comment #2, that __traits(allMembers) tries to expand the static foreach, and the static foreach calls __traits(allMembers), leading to unbounded mutual recursion and eventually a stack overflow. The exact same issue occurs if a template mixin in used in place of the string mixin in the above example. This makes some sense - how can you iterate over all members when the list isn't complete yet? For the sake of pragmatism though, it's probably sensible to allow iteration of the incomplete list, and certainly having the compiler crash is less than optimal.

This works now, test case added in https://github.com/dlang/dmd/pull/10016