Bug 18788 – static arrays with a length specified at runtime should dynamically allocate on the stack

Consider the following contrived example in C: ---main.c #include <stdio.h> #include <string.h> int writeln(char* s) { size_t len = strlen(s); size_t newLen = len + 1; char buf[newLen + 1]; // dynamically allocated on the stack memcpy(buf, s, len); buf[len] = '\n'; buf[newLen] = '\0'; fprintf(stdout, buf); } int main() { writeln("Hello, World!"); return 0; } I think D should do something similar with static arrays, but currently the following fails to compile: ---main.d import core.stdc.string; import core.stdc.stdio; void writeln(string s) { size_t len = s.length; size_t newLen = len + 1; char[newLen + 1] buf; // Error: variable newLen cannot be read at compile time memcpy(buf.ptr, s.ptr, len); buf[len] = '\n'; buf[newLen] = '\0'; fprintf(stdout, buf.ptr); } int main() { writeln("Hello, World!"); return 0; } Templates or mixins may be able to be used to workaround this limitation, but would ultimately result in code bloat if used extensively, and one of the use cases for this is in the domain of resource constrained microcontrollers. A possible implementation would be to have an alloca-equivalent druntime hook with hopefully a much better name, and have the compiler generate a call to it when appropriate. The d runtime hook could then forward to the platform's `alloca`, if one exists, or a custom D implementation could be implemented and optimized for each platform. Perhaps `scope` could also even play an interesting roll here.

this is the role of std.experimental.allocator.StackFront and friends (e.g. InSituRegion), there is probably room for runtime choice of StackFront and InSituRegion such that there is no stack allocation if the length exceeds the InSituRegion size. Perhaps the compiler should point users to this once it is made non-experimental, but this should not be a language feature as it is unsafe and may inadvertently blow the stack.

Yes it is unsafe, but not any less safe than void unsafeRecursion(size_t iterations) { if(iterations) { byte[256] buffer; unsafeRecursion(--iterations); } } As a compromise it could be restricted to @system code.

I'd actually just like to redefine `new` to mean the compiler is free to stack allocate it when it can prove the lifetime does not exceed that of the function (and the scope annotations can hint/prove/override this, as it already does for classes[!])

(In reply to Adam D. Ruppe from comment #3) > I'd actually just like to redefine `new` to mean the compiler is free to > stack allocate it when it can prove the lifetime does not exceed that of the > function (and the scope annotations can hint/prove/override this, as it > already does for classes[!]) Well, in principle, with DIP 1000, the case where scope on classes allocates the class on the stack simply becomes a compiler optimization rather than the specific purpose of scope (rather, the purpose of scope at that point is to give the compiler the information it needs to guarantee that no pointers or references or whatnot to the variable or object escape). As such, I don't see any reason why that optimization couldn't be expanded to cover other types allocated with new.

Another way of achieving the same result would be to make a `scope`d dynamic array allocate on the stack --- class C { } void main() { scope C c = new C(); // `c` is allocated on the stack scope int[] i = [1, 2, 3]; // `i` is inconsistently allocated on the heap } I think this should be consistent between classes and dynamic arrays; `i` should be allocated on the stack, and appending to it should allocate `realloc`ate it on the stack.

LDC has a pass[1] that does escape analysis for class instances and dynamic arrays and stack allocates them when possible. Sadly they do this on the LLVM IR level, so their solution can't be easily upstream-ed to dmd's frontend. Static arrays can't have their length specified at run-time, as it's part of their type. (Just like you can't specify template arguments at run-time. The only thing resembling providing compile-time arguments at run-time is simply instantiating various alternatives at compile-time and selecting one of them at run-time.) What you can do is an array-like struct that wraps alloca. Compile-time safety is out of the question, but it could try to check if it can fit in the remaining stack space minus a several pages. Even so, it would be quite brittle and heavily dependent on platform and run-time specifics (e.g. querying the fiber stack size, when not running on the native thread stack). AFAIK, `scope` is currently inferred only for value types (scalars and static arrays) with -dip1000. Perhaps the inference can be extended to cover dynamic arrays and classes as well. One thing that we must be careful about is that the array memory management [2] built on top of the GC has certain assumptions like looking for block metadata (e.g. array capacity) on its heap. These assumptions may fail if the compiler promotes the array on the stack. But if all of druntime's lifetime code is template-ized, in theory the compiler should be able to see through it and detect that the array .length getter property is trivially scope-friendly, while the setter is not. Don't know how .capacity should be handled. [1]: https://github.com/ldc-developers/ldc/blob/16f37466fe918e0824a3b2db239ba5a7ce5a33cc/gen/passes/GarbageCollect2Stack.cpp [2]: https://github.com/dlang/druntime/blob/bb7e1c0991c0ae43520e67c91d0b7b0d8d315d2e/src/rt/lifetime.d#L1442

(In reply to Mike Franklin from comment #0) > char[newLen + 1] buf; // Error: variable newLen cannot be read at > compile time I suspect the correct way to handle this would be: scope char[] buf = new char[newlen + 1]; The `scope` will ensure `buf` does not escape the stack frame, and so the compiler can allocate it on the stack. This is how: class C { ... } scope C c = new C(); works today.

(In reply to Mike Franklin from comment #5) > Another way of achieving the same result would be to make a `scope`d dynamic > array allocate on the stack > > --- > class C { } > > void main() > { > scope C c = new C(); // `c` is allocated on the stack > scope int[] i = [1, 2, 3]; // `i` is inconsistently allocated on the heap > } Ideally yes, but see discussion in issue 16037.

(In reply to Mike Franklin from comment #5) > Another way of achieving the same result would be to make a `scope`d dynamic > array allocate on the stack For the record, this got implemented in Issue 22306. Probably this can be closed now.

THIS ISSUE HAS BEEN MOVED TO GITHUB https://github.com/dlang/dmd/issues/19426 DO NOT COMMENT HERE ANYMORE, NOBODY WILL SEE IT, THIS ISSUE HAS BEEN MOVED TO GITHUB