Bug 15019 – [ICE] Heisencrash on OS X 32-bit with non-trivial projects

Created attachment 1547 Repro kit OS version: OS X 10.10.4 DMD version: 2.068.0 How to reproduce: Use ./build.sh in the attachment several time. DMD will occasionally crash with one of the following errors: - dmd(1328,0x7fff72416300) malloc: *** error for object 0x100a58c08: incorrect checksum for freed object - object was probably modified after being freed. - Assertion failed: (0), function merge2, file mtype.c, line 1701. - dmd failed with exit code -11. This is a heisenbug. I haven't reduced the testcase further since it tends to make the bug disappear or more rare.

Not reproducible on Linux. Why does build.sh have both distort.o and distort.d in DMD's command line? Perhaps that causes the result of successive runs to differ?

Created attachment 1549 Repro Kit removed distort.o from command-line It's a mistake that probably happened in reducing. I've removed the "distort.o" from build.sh and the bug still happen. Uploaded updated testcase.

Can't reproduce on Windows, where I had to remove the -fPIC flag.

Maybe you could try running DMD under Valgrind?

Here the valgrind output: http://pastebin.com/vMLUgbK1

Interestingly enough, DMD never crashes if launch through the lldb debugger.

Does -fPIC make any difference? Because on OS X PIC is always on.

Indeed it does not.

-g -debug and -w are not necessary to crash -m32 is necessary though

(In reply to ponce from comment #9) > -m32 is necessary though Out of curiosity, what's the use case for 32bit on OS X?

Some users in my target market (I don't know the exact proportion) keep using 32-bit audio plugins since if they upgrade to a 64-bit host they have to upgrade all their software plugins. Some of which do not exist in 64-bit. All desktop audio software support 32-bit Mac, to the exception of Apple Logic. LDC also has a bug that prevented 32-bit previously.

WIP here, will continue tomorrow: https://github.com/CyberShadow/dmd/commits/valgrind-20150912

I think I found the cause of the memory corruption, but not the root cause of the bug. If you apply these patches: https://github.com/CyberShadow/dmd/commit/2d424ac899535d975a6265b094c7f3fec217f4c5 https://github.com/CyberShadow/dmd/commit/0ea57be391b0a8307c1b5bf907506f74892ffe1d you will see that the assert fails with the given test case, although the DMD test suite passes with these asserts. It looks like DMD is trying to create a relocation outside of the data segment? Here's the backtrace: * frame #0: 0x00000001001d9a66 dmd`util_assert(file=0x0000000100322900, line=278) + 22 at util2.c:55 frame #1: 0x00000001001d072d dmd`local_assert(line=278) + 29 at tassert.h:29 frame #2: 0x00000001001d00d7 dmd`Outbuffer::setsize(this=0x0000000121e7d320, size=4100) + 55 at outbuf.c:278 frame #3: 0x000000010022e767 dmd`Obj::bytes(seg=2, offset=4100, nbytes=4, p=0x0000000100344f98) + 167 at machobj.c:2288 frame #4: 0x00000001001b93c6 dmd`addtofixlist(s=0x0000000100cf9b40, soffset=4100, seg=2, val=0, flags=16) + 374 at cgen.c:594 frame #5: 0x00000001002325b6 dmd`Obj::reftoident(seg=2, offset=4100, s=0x0000000100cf9b40, val=0, flags=16) + 150 at machobj.c:2469 frame #6: 0x000000010019c9c5 dmd`el_ptr(s=0x0000000100cf9b40) + 261 at el.c:1710 frame #7: 0x00000001001606ca dmd`toElem(this=0x00007fff5fbfd108, fe=0x0000000121120be0)::ToElemVisitor::visit(FuncExp*) + 138 at e2ir.c:1154 frame #8: 0x000000010008e986 dmd`FuncExp::accept(Visitor*) + 34 at expression.d:6324 frame #9: 0x000000010015a332 dmd`toElem(e=0x0000000121120be0, irs=0x00007fff5fbfd970) + 66 at e2ir.c:5485 frame #10: 0x000000010016254c dmd`toElem(this=0x00007fff5fbfd198, pe=0x000000012131e4d0)::ToElemVisitor::visit(PtrExp*) + 44 at e2ir.c:3690 frame #11: 0x000000010009ad0e dmd`PtrExp::accept(Visitor*) + 34 at expression.d:9738 frame #12: 0x000000010015a332 dmd`toElem(e=0x000000012131e4d0, irs=0x00007fff5fbfd970) + 66 at e2ir.c:5485 frame #13: 0x000000010016204b dmd`toElem(this=0x00007fff5fbfd328, ce=0x0000000121120b90)::ToElemVisitor::visit(CallExp*) + 1179 at e2ir.c:3605 frame #14: 0x0000000100099f16 dmd`CallExp::accept(Visitor*) + 34 at expression.d:9464 frame #15: 0x000000010015a332 dmd`toElem(e=0x0000000121120b90, irs=0x00007fff5fbfd970) + 66 at e2ir.c:5485 frame #16: 0x0000000100166625 dmd`toElem(this=0x00007fff5fbfd788, ae=0x00000001205dc9d0)::ToElemVisitor::visit(AssignExp*) + 8453 at e2ir.c:2949 frame #17: 0x00000001000a3476 dmd`AssignExp::accept(Visitor*) + 34 at expression.d:12155 frame #18: 0x000000010015a332 dmd`toElem(e=0x00000001205dc9d0, irs=0x00007fff5fbfd970) + 66 at e2ir.c:5485 frame #19: 0x000000010015b8a8 dmd`toElemDtor(e=0x00000001205dc9d0, irs=0x00007fff5fbfd970) + 88 at e2ir.c:5578 frame #20: 0x000000010014f3a2 dmd`S2irVisitor::visit(this=0x00007fff5fbfd890, s=0x0000000121120b00) + 130 at s2ir.c:846 frame #21: 0x000000010011aa73 dmd`ExpStatement::accept(Visitor*) + 31 at statement.d:1247 frame #22: 0x000000010014f242 dmd`Statement_toIR(s=0x0000000121120b00, irs=0x00007fff5fbfd970) + 66 at s2ir.c:1283 frame #23: 0x000000010014f446 dmd`S2irVisitor::visit(this=0x00007fff5fbfd930, s=0x0000000121121f10) + 134 at s2ir.c:862 frame #24: 0x000000010011bd83 dmd`CompoundStatement::accept(Visitor*) + 31 at statement.d:1587 frame #25: 0x000000010014f242 dmd`Statement_toIR(s=0x0000000121121f10, irs=0x00007fff5fbfd970) + 66 at s2ir.c:1283 frame #26: 0x000000010014f6b5 dmd`S2irVisitor::visit(this=0x00007fff5fbfda60, s=0x0000000121121e40) + 149 at s2ir.c:922 frame #27: 0x000000010011c4d3 dmd`ScopeStatement::accept(Visitor*) + 31 at statement.d:1763 frame #28: 0x000000010014f242 dmd`Statement_toIR(s=0x0000000121121e40, irs=0x00007fff5fbfdad0) + 66 at s2ir.c:1283 frame #29: 0x000000010014f588 dmd`S2irVisitor::visit(this=0x00007fff5fbfdbc0, s=0x0000000121121e60) + 280 at s2ir.c:895 frame #30: 0x000000010011c173 dmd`UnrolledLoopStatement::accept(Visitor*) + 31 at statement.d:1677 frame #31: 0x000000010014f242 dmd`Statement_toIR(s=0x0000000121121e60, irs=0x00007fff5fbfdc00) + 66 at s2ir.c:1283 frame #32: 0x000000010014f6b5 dmd`S2irVisitor::visit(this=0x00007fff5fbfdcf0, s=0x0000000121118060) + 149 at s2ir.c:922 frame #33: 0x000000010011c4d3 dmd`ScopeStatement::accept(Visitor*) + 31 at statement.d:1763 frame #34: 0x000000010014f242 dmd`Statement_toIR(s=0x0000000121118060, irs=0x00007fff5fbfdd58) + 66 at s2ir.c:1283 frame #35: 0x000000010014fd9b dmd`S2irVisitor::visit(this=0x00007fff5fbfde50, s=0x0000000121118020) + 331 at s2ir.c:198 frame #36: 0x0000000100123e23 dmd`IfStatement::accept(Visitor*) + 31 at statement.d:3287 frame #37: 0x000000010014f242 dmd`Statement_toIR(s=0x0000000121118020, irs=0x00007fff5fbfe328) + 66 at s2ir.c:1283 frame #38: 0x000000010014f446 dmd`S2irVisitor::visit(this=0x00007fff5fbfdef0, s=0x000000012111bea0) + 134 at s2ir.c:862 frame #39: 0x000000010011bd83 dmd`CompoundStatement::accept(Visitor*) + 31 at statement.d:1587 frame #40: 0x000000010014f242 dmd`Statement_toIR(s=0x000000012111bea0, irs=0x00007fff5fbfe328) + 66 at s2ir.c:1283 frame #41: 0x000000010014f446 dmd`S2irVisitor::visit(this=0x00007fff5fbfdf90, s=0x00000001213c7970) + 134 at s2ir.c:862 frame #42: 0x000000010011bd83 dmd`CompoundStatement::accept(Visitor*) + 31 at statement.d:1587 frame #43: 0x000000010014f242 dmd`Statement_toIR(s=0x00000001213c7970, irs=0x00007fff5fbfe328) + 66 at s2ir.c:1283 frame #44: 0x000000010014c679 dmd`FuncDeclaration_toObjFile(fd=0x00000001211166e0, multiobj=false) + 5433 at glue.c:1224 frame #45: 0x0000000100176af5 dmd`toObjFile(this=0x00007fff5fbfe510, fd=0x00000001211166e0)::ToObjFile::visit(FuncDeclaration*) + 37 at toobj.c:246 frame #46: 0x00000001000b679e dmd`FuncDeclaration::accept(Visitor*) + 34 at func.d:3721 frame #47: 0x000000010017427f dmd`toObjFile(this=0x00007fff5fbfe510, ti=0x00000001211162c0)::ToObjFile::visit(TemplateInstance*) + 207 at toobj.c:1155 frame #48: 0x000000010007940e dmd`TemplateInstance::accept(Visitor*) + 34 at dtemplate.d:7664 frame #49: 0x0000000100173b8f dmd`toObjFile(ds=0x00000001211162c0, multiobj=false) + 63 at toobj.c:1203 frame #50: 0x000000010014a10e dmd`genObjFile(m=0x00000001007069f0, multiobj=false) + 1118 at glue.c:385 frame #51: 0x00000001000dfafc dmd`tryMain(unsigned long, char const**) + 16368 at mars.d:1638 frame #52: 0x00000001000021bb dmd`_Dmain + 39 at mars.d:1697 frame #53: 0x00000001002491f4 dmd`D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv + 40 frame #54: 0x0000000100249139 dmd`D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ7tryExecMFMDFZvZv + 45 frame #55: 0x0000000100249199 dmd`D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZv + 45 frame #56: 0x0000000100249139 dmd`D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ7tryExecMFMDFZvZv + 45 frame #57: 0x00000001002490ac dmd`_d_run_main + 504 frame #58: 0x0000000100002216 dmd`main + 34 frame #59: 0x0000000100001668 dmd`_start + 230 frame #60: 0x0000000100001581 dmd`start + 33 Anyway, the assert should make the crash deterministic and it should be possible to further reduce the test case.

(In reply to Vladimir Panteleev from comment #13) > Anyway, the assert should make the crash deterministic and it should be > possible to further reduce the test case. Reduced: ///////////////////////////////// test.d //////////////////////////////// import std.string; struct Color() { static fromHex(char[] s) { import std.conv; s.to!ubyte; } } Color!() RGB ; struct Matrix(T, int R, int C) { Vector!(T, C) row_t; T[C] v; // all elements /// Covnerts to pretty string. string toString() const { try return format("%s", v); catch assert(false); // should not happen since format is right } } // GLSL is a big inspiration here // we defines types with more or less the same names template mat2x2(T) { Matrix!(T, 2, 2) mat2x2; } template mat3x3(T) { Matrix!(T, 3, 3) mat3x3; } template mat4x4(T) { Matrix!(T, 4, 4) mat4x4; } alias mat2x2 mat2; alias mat3x3 mat3; // shorter names for most common matrices alias mat4x4 mat4; string definePostfixAliases(string type) { return "alias " ~ type ~ "!byte " ~ type ~ "b;\n" "alias " ~ type ~ "!ubyte " ~ type ~ "ub;\n" "alias " ~ type ~ "!short " ~ type ~ "s;\n" "alias " ~ type ~ "!ushort " ~ type ~ "us;\n" "alias " ~ type ~ "!int " ~ type ~ "i;\n" "alias " ~ type ~ "!uint " ~ type ~ "ui;\n" "alias " ~ type ~ "!long " ~ type ~ "l;\n" "alias " ~ type ~ "!ulong " ~ type ~ "ul;\n" "alias " ~ type ~ "!float " ~ type ~ "f;\n" "alias " ~ type ~ "!double " ~ type ~ "d;\n"; } // define a lot of type names mixin(definePostfixAliases("mat2")); mixin(definePostfixAliases("mat3")); mixin(definePostfixAliases("mat4")); import std.string; struct Vector(T, int N) { T[N] v; string toString() { try return format("%s", v); catch assert(false); } } ///////////////////////////////////////////////////////////////////////// Trips the assert with: dmd -m32 -c all.d This is as far as I can go, though. Deferring to machobj.c's author, Walter Bright.

I applied the patches and cannot reproduce it :-(

(In reply to Walter Bright from comment #15) > I applied the patches and cannot reproduce it :-( So have you reproduced it, or can I do something to help?

Commits pushed to master at https://github.com/D-Programming-Language/dmd https://github.com/D-Programming-Language/dmd/commit/300f95cdad9dda22c98c62e4c09bef11926185d9 fix Issue 15019 - [ICE] Heisencrash on OS X 32-bit with non-trivial projects https://github.com/D-Programming-Language/dmd/commit/3b02f50cd97f82a5f0b302e3535e9a835a636442 Merge pull request #5074 from WalterBright/fix15019 fix Issue 15019 - [ICE] Heisencrash on OS X 32-bit with non-trivial projects