Bug 21089 – With vm.overcommit_memory=0, DMD can't link if it uses more than half the total (ram+swap) memory in the system.

Status
RESOLVED
Resolution
FIXED
Severity
minor
Priority
P1
Component
dmd
Product
D
Version
D2
Platform
x86
OS
Linux
Creation time
2020-07-29T13:30:35Z
Last change time
2020-08-03T12:16:08Z
Keywords
pull
Assigned to
No Owner
Creator
FeepingCreature

Comments

Comment #0 by default_357-line — 2020-07-29T13:30:35Z
This is because DMD fork()s to start the linker. Fork temporarily overcommit-allocates, and the allocation is refused by the system if `vm.overcommit_memory=0` and the forked DMD would push the total used address space over the total of memory + swap. This is a known issue, see the vfork() doc: * On systems where memory is constrained, vfork() avoids the need to temporarily commit memory (see the description of /proc/sys/vm/overcommit_memory in proc(5)) in order to execute a new program. (This can be especially beneficial where a large parent process wishes to execute a small helper program in a child process.) By contrast, using fork(2) in this scenario requires either committing an amount of memory equal to the size of the parent process (if strict overcommitting is in force) or overcommitting memory with the risk that a process is terminated by the out-of-memory (OOM) killer. But vfork() is itself deprecated. The fork call should be replaced with posix_spawn(), which does not share caller process memory.
Comment #1 by dlang-bot — 2020-07-29T14:21:23Z
@FeepingCreature created dlang/dmd pull request #11476 "Fix issue 21089: spawn linker with posix_spawn instead of fork()/exec()." fixing this issue: - Fix issue 21089: spawn linker with posix_spawn instead of fork()/exec(). This avoids an issue where Linux overcommit handling may block the fork() before it can execvp(). https://github.com/dlang/dmd/pull/11476
Comment #2 by dlang-bot — 2020-08-03T10:51:25Z
@FeepingCreature updated dlang/dmd pull request #11497 "Spawn linker with vfork to avoid need to overcommit memory" fixing this issue: - Fix issue 21089: spawn linker with vfork to avoid need to overcommit memory - dmd is large, ld is small. https://github.com/dlang/dmd/pull/11497
Comment #3 by dlang-bot — 2020-08-03T12:16:08Z
dlang/dmd pull request #11497 "Fix issue 21089: spawn linker with vfork to avoid need to overcommit memory" was merged into master: - c7aef70550c291928b20d3daa9bb4d643c460693 by Mathis Beer: Fix issue 21089: spawn linker with vfork to avoid need to overcommit memory - dmd is large, ld is small. https://github.com/dlang/dmd/pull/11497