Bug 23729 – ignore custom object.d for CTFE needs

Hello, I use a custom runtime because i need that for WASM and other reasons My runtime is minimal, it doesn't include bunch of stuff on purpose I encountered an issue on Linux only (on windows it compiles and runs fine) ``` // dmd -betterC import core.atomic; int count; extern(C) void main() { atomicStore(count, 0); count = (atomicFetchAdd(count, 1) + 1); count = (atomicFetchSub(count, 1) - 1); } ``` My custom object.d: https://gist.github.com/ryuukk/ecd85032b536e431d1350afbf945c3ad Windows: all good Linux: ``` dmd -betterC app.d -- using custom runtime -- /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(1107): Error: `object._d_arrayappendT` not found. The current runtime does not support appending array to arrays, or the runtime is corrupt. /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(1112): Error: `object._d_arrayappendcTXImpl` not found. The current runtime does not support appending element to arrays, or the runtime is corrupt. /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(1121): Error: `object._d_arrayappendT` not found. The current runtime does not support appending array to arrays, or the runtime is corrupt. /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(1137): Error: `object._d_arrayappendT` not found. The current runtime does not support appending array to arrays, or the runtime is corrupt. /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(324): called from here: `simpleFormat("\n asm pure nothrow @nogc @trusted\n {\n naked;\n xchg [%0], %1;\n ?2 mov %2, %1;\n ret;\n }\n ", ((string[3] __arrayliteral_on_stack1 = ["RSI", "EDI", null];) , cast(string[])__arrayliteral_on_stack1))` /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(233): Error: template instance `core.internal.atomic.atomicExchange!(MemoryOrder.seq, false, int)` error instantiating /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/atomic.d(127): instantiated from here: `atomicStore!(MemoryOrder.seq, int)` app.d(6): instantiated from here: `atomicStore!(MemoryOrder.seq, int, int)` /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(271): Error: CTFE failed because of previous errors in `simpleFormat` /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/atomic.d(172): Error: template instance `core.internal.atomic.atomicFetchAdd!(MemoryOrder.seq, true, int)` error instantiating app.d(7): instantiated from here: `atomicFetchAdd!(MemoryOrder.seq, int)` /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/atomic.d(201): Error: template instance `core.internal.atomic.atomicFetchSub!(MemoryOrder.seq, true, int)` error instantiating app.d(8): instantiated from here: `atomicFetchSub!(MemoryOrder.seq, int)` (dmd-2.102.1) ```

This is similar to issue 23726, where I mentioned: > This is a consequence of druntime hooks being translated to templates > in the front end: https://github.com/dlang/projects/issues/25 dmd and druntime are tied together. If you run a custom runtime without a hook for appending arrays, then it isn't surprising you get "Error: `object._d_arrayappendT` not found. The current runtime does not support appending array to arrays, or the runtime is corrupt."

That's shame.., dmd should use its runtime, not mine Specially when i compile with -betterC wich the compiled is supposed to deal with it without a runtime This should be treated as a bug

(In reply to ryuukk_ from comment #2) > That's shame.., dmd should use its runtime, not mine > > Specially when i compile with -betterC wich the compiled is supposed to deal > with it without a runtime > > > This should be treated as a bug What version of the compiler are you using? I cannot reproduce this.

DMD64 D Compiler v2.102.1 Installed using the installer script on Ubuntu The bug is only available on Linux, on Windows it compiles and runs fine

(In reply to ryuukk_ from comment #4) > DMD64 D Compiler v2.102.1 > > Installed using the installer script on Ubuntu > > The bug is only available on Linux, on Windows it compiles and runs fine So, if I understand the issue correctly, you are trying to build a program using your custom object.d integrated in the broader druntime? Or are you trying to compile your app.d with -betterC and then you want to link it with your own druntime? Please clarify. Also, the title is misleading, where is CTFE involved? You have provided a program that uses functions in core.atomic. This program is going to use the import paths that you are providing. If you are using a custom druntime, it will use that one. If the function is missing from your druntime, then obviously is going to fail. I don't see how we can fix this. You can compile with betterC and provide the import paths from the normal druntime and then when you link at runtime, you can link with your custom one. But again, this is not a bug in the compiler, it is a misconfiguration issue on the user side.

All information is provided in my 1st comment Take this code: ``` // dmd -betterC import core.atomic; int count; extern(C) void main() { atomicStore(count, 0); count = (atomicFetchAdd(count, 1) + 1); count = (atomicFetchSub(count, 1) - 1); } ``` create a file main.d, copy/paste the content then create a object.d file next to main.d and copy/paste the content from [1] then compile: ``dmd -betterC main.d`` on linux you'll get an error (check bellow) dmd shouldn't use my object.d, it has nothing to do with COMPILING the program, therefore it is a bug ``` dmd -betterC app.d -- using custom runtime -- /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(1107): Error: `object._d_arrayappendT` not found. The current runtime does not support appending array to arrays, or the runtime is corrupt. /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(1112): Error: `object._d_arrayappendcTXImpl` not found. The current runtime does not support appending element to arrays, or the runtime is corrupt. /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(1121): Error: `object._d_arrayappendT` not found. The current runtime does not support appending array to arrays, or the runtime is corrupt. /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(1137): Error: `object._d_arrayappendT` not found. The current runtime does not support appending array to arrays, or the runtime is corrupt. /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(324): called from here: `simpleFormat("\n asm pure nothrow @nogc @trusted\n {\n naked;\n xchg [%0], %1;\n ?2 mov %2, %1;\n ret;\n }\n ", ((string[3] __arrayliteral_on_stack1 = ["RSI", "EDI", null];) , cast(string[])__arrayliteral_on_stack1))` /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(233): Error: template instance `core.internal.atomic.atomicExchange!(MemoryOrder.seq, false, int)` error instantiating /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/atomic.d(127): instantiated from here: `atomicStore!(MemoryOrder.seq, int)` app.d(6): instantiated from here: `atomicStore!(MemoryOrder.seq, int, int)` /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/internal/atomic.d(271): Error: CTFE failed because of previous errors in `simpleFormat` /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/atomic.d(172): Error: template instance `core.internal.atomic.atomicFetchAdd!(MemoryOrder.seq, true, int)` error instantiating app.d(7): instantiated from here: `atomicFetchAdd!(MemoryOrder.seq, int)` /home/ryuukk/dlang/dmd-2.102.1/linux/bin64/../../src/druntime/import/core/atomic.d(201): Error: template instance `core.internal.atomic.atomicFetchSub!(MemoryOrder.seq, true, int)` error instantiating app.d(8): instantiated from here: `atomicFetchSub!(MemoryOrder.seq, int)` (dmd-2.102.1) ``` [1] - https://gist.github.com/ryuukk/ecd85032b536e431d1350afbf945c3ad

The problem is CTFE because the atomics module call this function: https://github.com/dlang/dmd/blob/d344030e01688fbe99b82438f4dae8dfa3f6aee3/druntime/src/core/internal/atomic.d#L1097 Wich does ``string append`` wich is not available in -betterC So it is a bug in the compiler, because he don't understand that DMD should use its object.d, not mine, i use -betterC, i don't have string append

The format function is used in a mixin context, therefore it's compile time https://github.com/dlang/dmd/blob/d344030e01688fbe99b82438f4dae8dfa3f6aee3/druntime/src/core/internal/atomic.d#L90

Here a minimal reduced reproducer: ``` --- main.d extern(C) void main() { mixin (simpleFormat(q{ asm pure nothrow @nogc @trusted { naked; ret; } }, [])); } string simpleFormat(string format, scope string[] args) { string result; outer: while (format.length) { foreach (i; 0 .. format.length) { if (format[i] == '%' || format[i] == '?') { bool isQ = format[i] == '?'; result ~= format[0 .. i++]; assert (i < format.length, "Invalid format string"); if (format[i] == '%' || format[i] == '?') { assert(!isQ, "Invalid format string"); result ~= format[i++]; } else { int index = 0; assert (format[i] >= '0' && format[i] <= '9', "Invalid format string"); while (i < format.length && format[i] >= '0' && format[i] <= '9') index = index * 10 + (ubyte(format[i++]) - ubyte('0')); if (!isQ) result ~= args[index]; else if (!args[index]) { size_t j = i; for (; j < format.length;) { if (format[j++] == '\n') break; } i = j; } } format = format[i .. $]; continue outer; } } result ~= format; break; } return result; } --- object.d module object; pragma(msg, "-- using custom runtime --"); alias size_t = typeof(int.sizeof); alias ptrdiff_t = typeof(cast(void*)0 - cast(void*)0); alias noreturn = typeof(*null); alias string = immutable(char)[]; alias wstring = immutable(wchar)[]; alias dstring = immutable(dchar)[];

> dmd shouldn't use my object.d, it has nothing to do with COMPILING the program, > therefore it is a bug The druntime and phobos sources are not hardwired into the compiler, they are found using the same lookup mechanism as other d source files, where the current working directory is the first import path. If you don't want the compiler to use your own object.d as the implicitly imported module and source of druntime hooks, then you should name it something else. However, it's clear that you fully intend to replace object.d because you want a custom runtime, so that's not the solution. > So it is a bug in the compiler, because he don't understand that DMD should > use its object.d, not mine Okay, let's say we add a special rule that object.d is only found in the installation's druntime folder. Then you don't have your custom lightweight runtime anymore, and all WASM incompatible dependencies are still dragged in. So what you really want is that your own object.d is used for 'code generation purposes', and the official object.d is used for 'CTFE purposes'. That's completely intractable. No compiler developer, let alone D user, is going to understand how that works. I don't see anything actionable in this issue, apart from making the druntime more pay as you go. That has long been a goal already, and would obviate the need for a custom lightweight druntime in the first place. But that's not a bug you can fix in an afternoon.

As stated before, this is not a bug. The compiler is working as intended, it's just that you are replacing the custom object.d with different one. If you want the compiler to not take into account your implementation of object, just name it something else, or put it in a different directory.