Bug 18461 – codegen bug - OPbt expressions and assignments to ambiguous symbols

This code should return exception in debug version or -4 exit code in release version. But in release version sometimes (50/50) it isn't fails. Faced this in another situation and reduced code to this sample. Also, tried to replace it with writeln(bt_result); and this issue is gone. /++ dub.sdl: name "bitop_bt" description "test" +/ import core.bitop; void main() { size_t test_val = 0b0001_0000; if(bt(&test_val, 4) != 0) assert(false); } How to run: $ dub run --single bitop_bt.d --build=release Output: denizzz@localhost:~/Dev$ dub run --single bitop_bt.d --build=release Performing "release" build using /usr/bin/dmd for x86_64. bitop_bt ~master: target for configuration "application" is up to date. To force a rebuild of up-to-date targets, run again with --force. Running ./bitop_bt denizzz@localhost:~/Dev$ dub run --single bitop_bt.d --build=release Performing "release" build using /usr/bin/dmd for x86_64. bitop_bt ~master: building configuration "application"... Linking... Running ./bitop_bt Program exited with code -4 denizzz@localhost:~/Dev$ dub run --single bitop_bt.d --build=release Performing "release" build using /usr/bin/dmd for x86_64. bitop_bt ~master: building configuration "application"... Linking... Running ./bitop_bt denizzz@localhost:~/Dev$ dub run --single bitop_bt.d --build=release Performing "release" build using /usr/bin/dmd for x86_64. bitop_bt ~master: target for configuration "application" is up to date. To force a rebuild of up-to-date targets, run again with --force. Running ./bitop_bt denizzz@localhost:~/Dev$ dub run --single bitop_bt.d --build=release Performing "release" build using /usr/bin/dmd for x86_64. bitop_bt ~master: building configuration "application"... Linking... Running ./bitop_bt Program exited with code -4 denizzz@localhost:~/Dev$ dub run --single bitop_bt.d --build=release Performing "release" build using /usr/bin/dmd for x86_64. bitop_bt ~master: target for configuration "application" is up to date. To force a rebuild of up-to-date targets, run again with --force. Running ./bitop_bt Program exited with code -4 denizzz@localhost:~/Dev$ dub run --single bitop_bt.d --build=release Performing "release" build using /usr/bin/dmd for x86_64. bitop_bt ~master: building configuration "application"... Linking... Running ./bitop_bt denizzz@localhost:~/Dev$ dub run --single bitop_bt.d --build=release Performing "release" build using /usr/bin/dmd for x86_64. bitop_bt ~master: target for configuration "application" is up to date. To force a rebuild of up-to-date targets, run again with --force. Running ./bitop_bt Program exited with code -4 denizzz@localhost:~/Dev$ dub run --single bitop_bt.d --build=release Performing "release" build using /usr/bin/dmd for x86_64. bitop_bt ~master: building configuration "application"... Linking... Running ./bitop_bt denizzz@localhost:~/Dev$ My system is 64bit Intel. $ dmd --version DMD64 D Compiler v2.078.2 Copyright (c) 1999-2017 by The D Language Foundation written by Walter Bright $ dub --version DUB version 1.7.2, built on Feb 7 2018

It is important to clarify: please run example several times, 10-20 times or so and you will see difference in exit code values.

I wonder whether this has something to do with DUB (probably not). 1) Can you reproduce the same with just running DMD? (I couldn't on my machine) --- dmd -release -run - <<EOF import core.bitop; void main() { size_t test_val = 0b0001_0000; if(bt(&test_val, 4) != 0) assert(false); } EOF --- 2) Did you check whether it's maybe something with your standard library? A simple check would be to try including core.bitop.bt in your file: https://github.com/dlang/druntime/blob/master/src/core/bitop.d#L270

> It is important to clarify: please run example several times, 10-20 times or so and you will see difference in exit code values. Tried it 100x times without any change: --- for i in $(seq 100) ; do dmd -release -run - <<EOF import core.bitop; void main() { size_t test_val = 0b0001_0000; if(bt(&test_val, 4) != 0) assert(false); } EOF done --- But I can reproduce it with dub: --- > seq 10 | parallel -j1 -n0 dub -v run --single bitop.d --build=release 2>&1 | grep "exited with code -4" Program exited with code -4 Program exited with code -4 Program exited with code -4 Program exited with code -4 ---

(In reply to Seb from comment #2) > I wonder whether this has something to do with DUB (probably not). > > 1) Can you reproduce the same with just running DMD? (I couldn't on my > machine) > > > --- > dmd -release -run - <<EOF [...] > --- You're missing `-O -inline`. DUB's release mode implies them. > 2) Did you check whether it's maybe something with your standard library? > A simple check would be to try including core.bitop.bt in your file: > > https://github.com/dlang/druntime/blob/master/src/core/bitop.d#L270 Same behavior.

Here's the verbose log from a failed dub build: --- Using dub registry url 'https://code.dlang.org/' Refreshing local packages (refresh existing: true)... Looking for local package map at /var/lib/dub/packages/local-packages.json Looking for local package map at /home/seb/.dub/packages/local-packages.json Try to load local package map at /home/seb/.dub/packages/local-packages.json Ignoring version specification (>=0.0.0) for path based dependency . Ignoring version specification (>=0.0.0) for path based dependency . Ignoring version specification (>=0.0.0) for path based dependency . Ignoring version specification (>=0.0.0) for path based dependency . Ignoring version specification (>=0.0.0) for path based dependency . Ignoring version specification (>=0.0.0) for path based dependency . Ignoring version specification (>=0.0.0) for path based dependency . Determined package version using GIT: dlang-tour 1.0.13+commit.19.gb19a18c Refreshing local packages (refresh existing: false)... Looking for local package map at /var/lib/dub/packages/local-packages.json Looking for local package map at /home/seb/.dub/packages/local-packages.json Try to load local package map at /home/seb/.dub/packages/local-packages.json Refreshing local packages (refresh existing: false)... Looking for local package map at /var/lib/dub/packages/local-packages.json Looking for local package map at /home/seb/.dub/packages/local-packages.json Try to load local package map at /home/seb/.dub/packages/local-packages.json Generating using build Generate target bitop_bt (executable /home/seb/dlang/dlang-tour/core bitop_bt) File '.' modified, need rebuild. /usr/bin/dmd -c -of.dub/build/application-release-linux.posix-x86_64-dmd_2078-B8815426D54654557310BC2B463B825C/bitop_bt.o -release -inline -O -w -version=Have_bitop_bt -Jviews bitop.d -vcolumns /usr/bin/dmd -of.dub/build/application-release-linux.posix-x86_64-dmd_2078-B8815426D54654557310BC2B463B825C/bitop_bt .dub/build/application-release-linux.posix-x86_64-dmd_2078-B8815426D54654557310BC2B463B825C/bitop_bt.o -L--no-as-needed Copying target from /home/seb/dlang/dlang-tour/core/.dub/build/application-release-linux.posix-x86_64-dmd_2078-B8815426D54654557310BC2B463B825C/bitop_bt to /home/seb/dlang/dlang-tour/core --- and indeed running ./bitop_bt locally doesn't result in an error. --- for i in $(seq 100) ; do echo "Running" dmd -release -run - <<EOF import core.bitop; void main() { size_t test_val = 0b0001_0000; if(bt(&test_val, 4) != 0) assert(false); } EOF done --- has this nice --- Running Error: program killed by signal 4 Running Error: program killed by signal 4 Running Error: program killed by signal 4 Running Error: program killed by signal 4 Running Error: program killed by signal 4 Running Error: program killed by signal 4 Running Error: program killed by signal 4 Running Error: program killed by signal 4 Running Error: program killed by signal 4 Running Error: program killed by signal 4 Running --- pattern. whereas adding -O -inline too, allows to reproduce your errors --- Running Running Running Running Error: program killed by signal 4 Running Running Running ---

The object files produced by DMD are identical and in fact since a few releases DMD produced reproducible builds. However, that's the problem: --- > dmd -inline -O -release bitop.d > ./bitop illegal hardware instruction (core dumped) ./bitop > ./bitop illegal hardware instruction (core dumped) ./bitop > ./bitop illegal hardware instruction (core dumped) ./bitop > ./bitop illegal hardware instruction (core dumped) ./bitop > ./bitop <-- works fine ---

Good code generated with `-O -inline` (no `-release`): ---- 0000000000000000 <_Dmain>: 0: 55 push rbp 1: 48 8b ec mov rbp,rsp 4: 48 83 ec 10 sub rsp,0x10 8: 48 c7 45 f8 10 00 00 mov QWORD PTR [rbp-0x8],0x10 f: 00 10: 48 8d 45 f8 lea rax,[rbp-0x8] 14: 48 0f ba 20 04 bt QWORD PTR [rax],0x4 ---- Bad code with `-release -O -inline`: ---- 0000000000000000 <_Dmain>: 0: 55 push rbp 1: 48 8b ec mov rbp,rsp 4: 48 83 ec 10 sub rsp,0x10 8: 48 8d 45 f8 lea rax,[rbp-0x8] c: 48 0f ba 20 04 bt QWORD PTR [rax],0x4 ---- Note that `mov QWORD PTR [rbp-0x8],0x10` is missing in the bad code. I.e., `test_val` doesn't get initialized.

> Tried it 100x times without any change: Reproduceable if -O and -inline switches used both

This is a (major?) codegen bug in dmd, the joint action of -inline and -release makes the rmdeadass kill the `test_val' definition and fold the argument `p' of the `bt' function into `&test_val': that's right, the variable is gone but the reference to it is still there, leading to the generation of the silly `lea' instruction pointing into the stack. Some more details for the lovers of the gory details: ``` ambiguous lvalue: test_val(0) = 16LL assnod[0]: test_val(0) = 16LL POSS DEAD=0, live=0 dead assignment (test_val(0) = 16LL ) Boutlv elimass el:0x55687ec1d360 cnt=0 = TYuns long long 0x55687ec1d170 0x55687ec1d300 el:0x55687ec1d170 cnt=0 var TYuns long long test_val el:0x55687ec1d300 cnt=0 const TYuns long long 16LL assnod[1]: p(1) = #test_val(0) POSS DEAD=0, live=0 dead assignment (p(1) = #test_val(0)) Boutlv elimass el:0x55687ec1d9f0 cnt=0 = mTYconst|TY* 0x55687ec1d810 0x55687ec1d8d0 el:0x55687ec1d810 cnt=0 var mTYconst|TY* p el:0x55687ec1d8d0 cnt=0 relconst TY* 0+& test_val assnod[2]: bitnum(2) = 4LL POSS DEAD=0, live=0 dead assignment (bitnum(2) = 4LL ) Boutlv elimass el:0x55687ec1dcc0 cnt=0 = TYuns long long 0x55687ec1dba0 0x55687ec1dc60 el:0x55687ec1dba0 cnt=0 var TYuns long long bitnum el:0x55687ec1dc60 cnt=0 const TYuns long long 4LL ``` If I understand correctly what's going on the variables are killed even though they're not dead because they don't escape the block (they're not in Boutlv).

Commits pushed to master at https://github.com/dlang/dmd https://github.com/dlang/dmd/commit/85251fa4bc1907f5ecee1106f2cca9cb1f023d11 fix Issue 18461 - codegen bug - OPbt expressions and assignments to ambiguous symbols https://github.com/dlang/dmd/commit/74226cfab38a6e32f78a877af4a2a89687aac348 Merge pull request #8065 from WalterBright/fix18461 fix Issue 18461 - codegen bug - OPbt expressions and assignments to a… merged-on-behalf-of: Iain Buclaw <[email protected]>