Bug 24293 – ImportC: C preprocessor output should use temporary files

STR: ---- 1. Have /usr/include/mysql/mysql.h installed in your system 2. run this bash script: ``` #!/bin/bash set -CEeuo pipefail #set -x cat >| cmysql.c <<'_' #include <mysql/mysql.h> _ for dst in {1..9} do cat >| xmysqltest${dst}.d <<'_' import std.stdio; import cmysql; int main () { return 0; } _ done : ${CD:=dmd} for dst in {1..9} do ${CD} xmysqltest${dst}.d & pids+=($!) done rc=0 for pid in "${pids[@]}" do rv=0 wait $pid || rv=$? ((rc |= $rv)) || : echo "exit code of process with pid <$pid>: <$rv>" done exit $rc ``` found: ------ xmysqltest4.d(2): Error: cannot find input file `cmysql.c` import path[0] = [...]/dmd2/linux/bin64/../../src/phobos import path[1] = [...]/dmd2/linux/bin64/../../src/druntime/import xmysqltest7.d(2): Error: cannot find input file `cmysql.c` import path[0] = [...]/dmd2/linux/bin64/../../src/phobos import path[1] = [...]/dmd2/linux/bin64/../../src/druntime/import /usr/include/bits/pthreadtypes.h(23): Error: unterminated string constant starting at /usr/include/bits/pthreadtypes.h(23) xmysqltest9.d(2): Error: cannot find input file `cmysql.c` import path[0] = [...]/dmd2/linux/bin64/../../src/phobos import path[1] = [...]/dmd2/linux/bin64/../../src/druntime/import xmysqltest8.d(2): Error: cannot find input file `cmysql.c` import path[0] = [...]/dmd2/linux/bin64/../../src/phobos import path[1] = [...]/dmd2/linux/bin64/../../src/druntime/import exit code of process with pid <4843>: <0> exit code of process with pid <4844>: <0> exit code of process with pid <4845>: <0> exit code of process with pid <4846>: <1> exit code of process with pid <4847>: <0> exit code of process with pid <4848>: <1> exit code of process with pid <4849>: <1> exit code of process with pid <4850>: <1> exit code of process with pid <4851>: <1> expected: compile successfully

Try it serially.

(In reply to Walter Bright from comment #1) > Try it serially. I did so before and it worked (surprisingly!). But when it came to make -j4 the non-reentrancy of dmd showed up. The cause of the problem is, that for every import statement dmd invokes cpp to generate for each header file $h.h a preprocessed $h.i (pure C file). This can be prevented by creating the .i file with cpp -m64 -dD -Wno-builtin-macro-redefined cmysql.c -include [...]dmd2/linux/bin64/../../src/druntime/import/importc.h cmysql.i once and before starting the compilation of the dependent D source files. I would appreciate if dmd had a switch to generate the .i from the .c. A short note in the docs would be nice, too.

I'm not understanding the need for dmd to generate the .i file rather than cpp? I'm also not understanding the reentrancy problem - are multiple .i files being generated with the same base name, thus conflicting with each other?

(In reply to Walter Bright from comment #3) > I'm not understanding the need for dmd to generate the .i file rather than > cpp? dmd invokes cpp in order to generade the .i file: $ strace -s128 -feexecve dmd cmysql.c [...] [pid 24506] execve("/usr/bin/cpp", ["cpp", "-m64", "-dD", "-Wno-builtin-macro-redefined", "cmysql.c", "-include", "[...]/dmd2/linux/bin64/../../src/druntime/import/importc.h", "cmysql.i"], [/* 96 vars */]) = 0 [...] > I'm also not understanding the reentrancy problem - are multiple .i > files being generated with the same base name, thus conflicting with each > other? There are two or more processes each writing to a file with the same file name (cmysql.i): $ strace -s128 -feexecve dmd xmysqltest1.d [...] [pid 24560] execve("/usr/bin/cpp", ["cpp", "-m64", "-dD", "-Wno-builtin-macro-redefined", "cmysql.c", "-include", "[...]/dmd2/linux/bin64/../../src/druntime/import/importc.h", "cmysql.i"], [/* 96 vars */]) = 0 [...] $ strace -s128 -feexecve dmd xmysqltest2.d [...] [pid 24566] execve("/usr/bin/cpp", ["cpp", "-m64", "-dD", "-Wno-builtin-macro-redefined", "cmysql.c", "-include", "[...]/dmd2/linux/bin64/../../src/druntime/import/importc.h", "cmysql.i"], [/* 96 vars */]) = 0 [...] If the processes run in parallel cmysql.i refers to the same file. Since cmysql.i is opened with O_TRUNC the content already written is lost.

Ok, I think I see the problem. apple.d imports pear.c peach.d imports pear.c The commands: dmd -c apple.d dmd -c peach.d both started concurrently mean both try to write then read pear.i, stepping on each other. A possible solution would be to create a temporary file rather than pear.i.

@WalterBright created dlang/dmd pull request #15981 "fix Issue 24293 - ImportC: C preprocessor output should use temporary…" fixing this issue: - fix Issue 24293 - ImportC: C preprocessor output should use temporary files https://github.com/dlang/dmd/pull/15981

dlang/dmd pull request #15981 "fix Issue 24293 - ImportC: C preprocessor output should use temporary…" was merged into master: - 2abdce7fa11fa08a3f33a40206d31c70e2af6fdd by Walter Bright: fix Issue 24293 - ImportC: C preprocessor output should use temporary files https://github.com/dlang/dmd/pull/15981

*** Issue 24252 has been marked as a duplicate of this issue. ***