Bug 19513 – Use sched_getaffinity(2) to get the number of CPU cores if available

Status
RESOLVED
Resolution
FIXED
Severity
enhancement
Priority
P1
Component
phobos
Product
D
Version
D2
Platform
All
OS
Linux
Creation time
2018-12-25T10:56:33Z
Last change time
2019-05-28T16:23:21Z
Keywords
pull
Assigned to
No Owner
Creator
Hiroki Noda

Comments

Comment #0 by kubo39 — 2018-12-25T10:56:33Z
Currently std.parallelism.totalCPUs is implemented by sysconf(_SC_NPROCESSORS_ONLN). However, on GNU/Linux, usable number of processoes may be restricted if a process runs in containers. In case it's better to use sched_getaffinity(2). How to implement: 1. std.parallelism.totalCPUs uses sched_getaffinity(2) internally. 2. Add new API (and considering function name). - std.parallelism.sched_getaffinity(pid, mask)? (Like Python) - std.parallelism.nprocessors? (Like Ruby) - or berrer name Libraries exposing sched_getaffinity: - Python >= 3.3 provides os.get_affinity(pid, mask) function. - Ruby >= 2.2 provides Etc.nprocessors function. - GNU coreutils provides nproc(1) command. - Rust's num_cpus crate provides num_cpus::get() function. Links: - https://github.com/golang/go/issues/3921 - https://docs.python.org/3/library/os.html#os.sched_getaffinity - https://bugs.ruby-lang.org/issues/10267 - http://man7.org/linux/man-pages/man1/nproc.1.html
Comment #1 by dlang-bot — 2019-05-24T03:33:45Z
@kubo39 created dlang/phobos pull request #7030 "Fix Issue 19513 - Use sched_getaffinity(2) to get the number of CPU cores if abailable" fixing this issue: - Fix Issue 19513 - Use sched_getaffinity(2) to get the number of CPU cores if available Currently std.parallelism.totalCPUs is implemented by sysconf(_SC_NPROCESSORS_ONLN) on Posix platoform. However, on GNU/Linux, usable number of processors may be restricted if a process runs in container. In case it's better to use sched_getaffinity(2). ref: http://man7.org/linux/man-pages/man1/nproc.1.html https://github.com/dlang/phobos/pull/7030
Comment #2 by dlang-bot — 2019-05-28T16:23:21Z
dlang/phobos pull request #7030 "Fix Issue 19513 - Use sched_getaffinity(2) to get the number of CPU cores if abailable" was merged into master: - d17f2eb9f298b8b4957ee74603445d2211ff41d7 by Hiroki Noda: Fix Issue 19513 - Use sched_getaffinity(2) to get the number of CPU cores if available Currently std.parallelism.totalCPUs is implemented by sysconf(_SC_NPROCESSORS_ONLN) on Posix platoform. However, on GNU/Linux, usable number of processors may be restricted if a process runs in container. In case it's better to use sched_getaffinity(2). ref: http://man7.org/linux/man-pages/man1/nproc.1.html https://github.com/dlang/phobos/pull/7030