Bug 14258 – SIGSEGV in dpldocs

Created attachment 1481 gdb log When attempting to build the docs from a non-interactive shell, using a clean checkout and environment, dpldocs get stuck and eats all RAM. I get a different result (a SIGSEGV) if I run it under GDB in an interactive shell. See attached log. I'm guessing it gets stuck on a recursive template expansion, or something? This has been happening for a few weeks now, I think, and is currently blocking updating dlang.org/library/ (I've been procrastinating investigating/filing this).

I think it's having trouble with this expansion: LIX=$(LI $1)$(LIX $+) Its usage might have been introduced here: https://github.com/D-Programming-Language/phobos/commit/743303055d74cdd3e482a3af3e99af93755f3af6

(In reply to Vladimir Panteleev from comment #1) > I think it's having trouble with this expansion: > > LIX=$(LI $1)$(LIX $+) That idiom is used regularly.

Thanks for reducing this, will try to fix it soon.

I haven't actually reduced it, just added some debug logging to dpldocs and looked with git blame.

The crash occurs because the proper termination conditions for recursive macro calls are not implemented. I'll do that. There could be an additional issue in the GC/array implementation, though, as I would have expected an OutOfMemoryError instead of a segmentation fault (except if the segfault is caused by a stack overflow).

Except for address space exhaustion, OutOfMemoryError will never realistically occur in any OS with overcommit.

My systems all have limited swap space - what would the OS do when all that memory is filled with data (note that the array that is piled up is actually written to)? I've added the termination conditions now and will file a PR as soon as the new DDOX version is up on code.dlang.org.

On Linux, what usually happens upon attempting to commit (write to) a page that has been overcommitted (COW of a zero page), when there are no free pages and no pages to swap or reclaim, the OOM killer is invoked and the most memory-hungry process dies with a SIGSEGV.

Hm, you are right of course, totally forgot about that.

Commit pushed to master at https://github.com/D-Programming-Language/dlang.org https://github.com/D-Programming-Language/dlang.org/commit/cc8d8123ef0dff1705e729d1faad2b710639771f Merge pull request #925 from s-ludwig/master Issue 14258 - SIGSEGV in dpldocs

(In reply to github-bugzilla from comment #10) > Commit pushed to master at > https://github.com/D-Programming-Language/dlang.org > > https://github.com/D-Programming-Language/dlang.org/commit/ > cc8d8123ef0dff1705e729d1faad2b710639771f > Merge pull request #925 from s-ludwig/master > > Issue 14258 - SIGSEGV in dpldocs Does this mean this can be marked as fixed?

Yes, I confirmed it's fixed in git.

Commit pushed to stable at https://github.com/D-Programming-Language/dlang.org https://github.com/D-Programming-Language/dlang.org/commit/cc8d8123ef0dff1705e729d1faad2b710639771f Merge pull request #925 from s-ludwig/master