Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Added numa_support rfc #1535

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

Conversation

vossmjp
Copy link
Contributor

@vossmjp vossmjp commented Oct 23, 2024

Description

Adds RFC for simplified NUMA support

Fixes # - issue number(s) if exists

Type of change

Choose one or multiple, leave empty if none of the other choices apply

Add a respective label(s) to PR if you have permissions

  • bug fix - change that fixes an issue
  • new feature - change that adds functionality
  • tests - change in tests
  • infrastructure - change in infrastructure and CI
  • documentation - documentation update

Tests

  • added - required for new features and some bug fixes
  • not needed

Documentation

  • updated in # - add PR number
  • needs to be updated
  • not needed

Breaks backward compatibility

  • Yes
  • No
  • Unknown

Notify the following users

List users with @ to send notifications

Other information

Comment on lines 28 to 31
Below is the example that demonstrates the use of these APIs to pin threads to different
arenas to each of the NUMA nodes available on a system, submit work across those `task_arena`
objects and into associated `task_group`` objects, and then wait for work again using both
the `task_arena` and `task_group` objects.
Copy link
Contributor

@akukanov akukanov Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the code can be made simpler with std::thread per NUMA domain, instead of relying only on TBB. On the one hand, it also signals that TBB lacks high-level NUMA APIs. On the other hand, TBB, and task arenas specifically, were designed to work well with application level threads where it makes sense. I think it is much better to assume/suggest each NUMA aware arena to be used by a special application thread than to add extra levels of complication with task groups.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern of task_arenas and task_groups is what we show in our documentation: for example here. And, probably as a consequence, a pattern we see in applications that use NUMA constraints.

Copy link
Contributor

@akukanov akukanov Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the documentation shows a suboptimal pattern then. In particular, it does not explicitly set the number of reserved slots to 0, and essentially can lead to undersubscription. Why repeating the same mistake one more time? :)

@vossmjp vossmjp changed the title Added numa_support rfc [RFC] Added numa_support rfc Oct 30, 2024
Copy link
Contributor

@aleksei-fedotov aleksei-fedotov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few typos to fix.

rfcs/proposed/simplified_numa_support/README.md Outdated Show resolved Hide resolved
rfcs/proposed/simplified_numa_support/README.md Outdated Show resolved Hide resolved
rfcs/proposed/simplified_numa_support/README.md Outdated Show resolved Hide resolved
rfcs/proposed/simplified_numa_support/README.md Outdated Show resolved Hide resolved
rfcs/proposed/simplified_numa_support/README.md Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants