Bug 24239 – dlang.org tests on CircleCI run out of memory

Status
RESOLVED
Resolution
FIXED
Severity
normal
Priority
P1
Component
dlang.org
Product
D
Version
D2
Platform
x86_64
OS
Linux
Creation time
2023-11-11T12:02:39Z
Last change time
2023-11-14T19:47:24Z
Keywords
pull
Assigned to
No Owner
Creator
Tim

Comments

Comment #0 by tim.dlang — 2023-11-11T12:02:39Z
The tests for dlang.org running on CircleCI can randomly fail with the following message: Last command failed with exit code -9 In https://github.com/dlang/dmd/pull/15504 I found out, that build.d tries to run DMD in parallel with 36 instances. This number was automatically detected as the number of CPU cores by the function totalCPUs in build.d. The CircleCI documentation (https://circleci.com/docs/configuration-reference/#resourceclass) mentions this problem: "Java, Erlang and any other languages that introspect the /proc directory for information about CPU count may require additional configuration to prevent them from slowing down when using the CircleCI resource class feature. Programs with this issue may request 32 CPU cores and run slower than they would when requesting one core. Users of languages with this issue should pin their CPU count to their guaranteed CPU resources." Pull Request https://github.com/dlang/dmd/pull/15504 tried to better calculate the number of CPU cores, but that could result in a wrong value in other environments. An alternative is to manually set the number of jobs, which needs changes to the makefiles of dmd and dlang.org.
Comment #1 by dlang-bot — 2023-11-11T12:05:01Z
@tim-dlang created dlang/dmd pull request #15799 "Issue 24239 - dlang.org tests on CircleCI run out of memory" mentioning this issue: - Issue 24239 - dlang.org tests on CircleCI run out of memory Add variable BUILD_JOBS to posix.mak, which allows to set the number of jobs for build.d. This will be used by a separate pull request for dlang.org to manually set it, because the automatically calculated number of CPU cores is too high. https://github.com/dlang/dmd/pull/15799
Comment #2 by dlang-bot — 2023-11-11T12:07:22Z
@tim-dlang created dlang/dlang.org pull request #3724 "Fix issue 24239 - dlang.org tests on CircleCI run out of memory" fixing this issue: - Fix issue 24239 - dlang.org tests on CircleCI run out of memory Manually set the number of jobs used by build.d on CircleCI, because the automatically calculated number of CPU cores is too high. This depends on https://github.com/dlang/dmd/pull/15799. https://github.com/dlang/dlang.org/pull/3724
Comment #3 by dlang-bot — 2023-11-14T13:03:36Z
dlang/dmd pull request #15799 "Issue 24239 - dlang.org tests on CircleCI run out of memory" was merged into master: - 6d9e93681479e23a21c103d29a6d63c17930ab9a by Tim Schendekehl: Issue 24239 - dlang.org tests on CircleCI run out of memory Add variable BUILD_JOBS to posix.mak, which allows to set the number of jobs for build.d. This will be used by a separate pull request for dlang.org to manually set it, because the automatically calculated number of CPU cores is too high. https://github.com/dlang/dmd/pull/15799
Comment #4 by dlang-bot — 2023-11-14T19:47:24Z
dlang/dlang.org pull request #3724 "Fix issue 24239 - dlang.org tests on CircleCI run out of memory" was merged into master: - 43ca8489bdc23832782ef424a5f2cc27359fce83 by Tim Schendekehl: Fix issue 24239 - dlang.org tests on CircleCI run out of memory Manually set the number of jobs used by build.d on CircleCI, because the automatically calculated number of CPU cores is too high. This depends on https://github.com/dlang/dmd/pull/15799. https://github.com/dlang/dlang.org/pull/3724