Skip to content

unix: add (*CPUSet).Fill helper to enable all CPUs #259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

unix: add (*CPUSet).Fill helper to enable all CPUs #259

wants to merge 1 commit into from
+9 −0

Conversation

cyphar
Copy link
Contributor

@cyphar cyphar commented Aug 21, 2025 •
edited
Loading

Some programs (such as container runtimes) want to reset their CPU
affinity if they are spawned by processes with a particular CPU
affinity. Container runtimes didn't really have to deal with this issue
until Linux 6.2 when the cpuset cgroup was changed to no longer
auto-reset CPU affinity in this case.

A naive approach to resetting your CPU affinity would be to get the
number of CPUs by looking at "/proc/stat" or "/sys/devices/system/cpu"
(note that runtime.NumCPU() actually returns the CPU affinity of the
process at startup time, which isn't useful for this purpose) and then
asking for all of those CPUs.

However, sched_setaffinity(2) will silently ignore any CPU bits set in
the provided CPUSet if they do not exist or are not enabled in the
cpuset cgroup of the process. This means that you can reset your CPU
affinity by just setting every CPU bit in CPUSet and passing it to
sched_setaffinity(2).

Unfortunately, setting every CPU bit in CPUSet with (*CPUSet).Set() is
very inefficient. If it were possible to just memset(0xFF) the CPUSet
array, users would be able to reset their CPU affinity even more
cheaply. However, Go doesn't have a memset primitive that can be used in
that way.

Obvious solutions like setting the array elements of CPUSet to (^0) do
not work because CPUSet is an array of a private newtype and so the
compiler complains if you try to use a constant like (^0) without a
cast (and we cannot use a cast because the type is private):

cannot use ^0 (untyped int constant -1) as
"golang.org/x/sys/unix".cpuMask value in assignment (overflows)

The only real alternative is to do something quite hacky like:

cpuset := unix.CPUSet{}
for i := range cpuset {
    cpuset[i]-- // underflow to 0xFF..FF
}

... which is the solution we use in runc.

It would be much nicer to have a helper that does this memset for us in
a less hacky way, since resetting CPU affinity seems like a fairly
common operation.

Ref: Linux kernel commit da019032819a ("sched: Enforce user requested affinity")

@cyphar cyphar mentioned this pull request Aug 21, 2025
Merged
@gopherbot
Copy link
Contributor

This PR (HEAD: 9f87bcc) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/sys/+/698015.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot
Copy link
Contributor

Message from Gopher Robot:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

This PR (HEAD: 79c52c7) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/sys/+/698015.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot
Copy link
Contributor

Message from Aleksa Sarai:

Patch Set 2:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@cyphar
unix: add (*CPUSet).Fill helper to enable all CPUs
71871db 
Some programs (such as container runtimes) want to reset their CPU
affinity if they are spawned by processes with a particular CPU
affinity. Container runtimes didn't really have to deal with this issue
until Linux 6.2 when the cpuset cgroup was changed to no longer
auto-reset CPU affinity in this case.

A naive approach to resetting your CPU affinity would be to get the
number of CPUs by looking at "/proc/stat" or "/sys/devices/system/cpu"
(note that runtime.NumCPU() actually returns the CPU affinity of the
process at startup time, which isn't useful for this purpose) and then
asking for all of those CPUs.

However, sched_setaffinity(2) will silently ignore any CPU bits set in
the provided CPUSet if they do not exist or are not enabled in the
cpuset cgroup of the process. Unfortunately, setting every possible CPU
bit in CPUSet with (*CPUSet).Set() is very inefficient.

If it were possible to just memset(0xFF) the CPUSet array, users would
be able to reset their CPU affinity very cheaply. However, Go doesn't
have a memset primitive that can be used in that way.

Obvious solutions like setting the array elements of CPUSet to (^0) do
not work because CPUSet is an array of a private newtype and so the
compiler complains if you try to use a constant like (^0) without a
cast (and we cannot use a cast because the type is private):

  cannot use ^0 (untyped int constant -1) as "golang.org/x/sys/unix".cpuMask value in assignment (overflows)

The only real alternative is to do something quite hacky like:

  cpuset := unix.CPUSet{}
  for i := range cpuset {
      cpuset[i]-- // underflow to 0xFF..FF
  }

... which is the solution we use in runc.

It would be much nicer to have a helper that does this memset for us in
a less hacky way, since resetting CPU affinity seems like a fairly
common operation.

Ref: Linux kernel commit da019032819a ("sched: Enforce user requested affinity")
Signed-off-by: Aleksa Sarai <[email protected]>
@gopherbot
Copy link
Contributor

Message from Aleksa Sarai:

Patch Set 3:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

This PR (HEAD: 71871db) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/sys/+/698015.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot
Copy link
Contributor

Message from Aleksa Sarai:

Patch Set 5:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Kirill Kolyshkin:

Patch Set 5: Code-Review+2 Commit-Queue+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Go LUCI:

Patch Set 5:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2025-08-28T00:31:00Z","revision":"a946a41b7fba35fb245539fa405fc58b37629cdd"}

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Kirill Kolyshkin:

Patch Set 5: -Commit-Queue

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Go LUCI:

Patch Set 5:

This CL has passed the run

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Go LUCI:

Patch Set 5: LUCI-TryBot-Result+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Cherry Mui:

Patch Set 5:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Kirill Kolyshkin:

Patch Set 5:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Cherry Mui:

Patch Set 5:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/698015.
After addressing review feedback, remember to publish your drafts!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@cyphar @gopherbot