Bug 3208 – setAssertHandler leads to segfault

The following code produces a segmentation fault. If compiled with -g this does not occur. import core.exception; void handleAssertion(string file, size_t line, string msg = null) {}; static this() { setAssertHandler( &handleAssertion ); } void main() { assert(false); }

My guess is that the compiler generates code, that doesn't expect assert(false); to actually return normally. The assert(false) statement generates the following function, which apparently does the job of actually calling _d_assert, if the assertion condition wasn't fulfilled: 080495a8 <_D2rt8__assertFiZv>: 80495a8: 55 push ebp 80495a9: 8b ec mov ebp,esp 80495ab: 50 push eax 80495ac: ff 35 98 6a 06 08 push DWORD PTR ds:0x8066a98 80495b2: ff 35 94 6a 06 08 push DWORD PTR ds:0x8066a94 80495b8: e8 a7 8e 00 00 call 8052464 <_d_assert> 80495bd: 5d pop ebp 80495be: c3 ret As you can see, the stack after the call _d_assert isn't cleaned up. There are still 3 parameters on the stack. As far as I know, it's the caller's job to clean up (_d_assert is declared as extern(C) and this is on Linux). This means the ret statement would jump to a bogus memory position. But I could be wrong.

(In reply to comment #1) > My guess is that the compiler generates code, that doesn't expect > assert(false); to actually return normally. You're probably right. In my investigation I found the only way to *not* have it crash was to throw an exception from the assertion handler.

Yeah, DMD requires the assert handler to throw. It's not an optimization I'm particularly fond of, but it doesn't seem likely that this will change. Since this isn't a library issue, I suggest either closing the ticket or redirecting it to Walter as an enhancement request for DMD.

Allright, I'd like to propose this as an enhancement.

This now works but setAssertHandler is deprecated, heh: // import core.exception; nothrow void handleAssertion(string file, size_t line, string msg = null) {} static this() { setAssertHandler( &handleAssertion ); } void main() { assert(false); }