-
Notifications
You must be signed in to change notification settings - Fork 190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds support for large number of items to DeviceScan::*ByKey
family of algorithms
#2477
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
elstehle
commented
Sep 28, 2024
|
||
using ScanTileStateT = ReduceByKeyScanTileState<AccumT, OffsetT>; | ||
using ScanTileStateT = ReduceByKeyScanTileState<AccumT, int>; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had tried making this bool
but performance dropped for some workloads by 20%
. Using int
with the logical or, |
, operator conserved both the semantics and performance, and made the algorithms performance almost agnostic to the offset type.
🟩 CI finished in 1h 30m: Pass: 100%/208 | Total: 5d 13h | Avg: 38m 39s | Max: 1h 05m | Hits: 63%/14066
|
Project | |
---|---|
CCCL Infrastructure | |
libcu++ | |
+/- | CUB |
Thrust | |
CUDA Experimental | |
pycuda | |
CUDA C Core Library |
Modifications in project or dependencies?
Project | |
---|---|
CCCL Infrastructure | |
libcu++ | |
+/- | CUB |
+/- | Thrust |
CUDA Experimental | |
+/- | pycuda |
+/- | CUDA C Core Library |
🏃 Runner counts (total jobs: 208)
# | Runner |
---|---|
171 | linux-amd64-cpu16 |
16 | linux-arm64-cpu16 |
12 | linux-amd64-gpu-v100-latest-1 |
9 | windows-amd64-cpu16 |
🟩 CI finished in 1h 29m: Pass: 100%/208 | Total: 5d 14h | Avg: 38m 49s | Max: 1h 09m | Hits: 63%/14066
|
Project | |
---|---|
CCCL Infrastructure | |
libcu++ | |
+/- | CUB |
Thrust | |
CUDA Experimental | |
pycuda | |
CUDA C Core Library |
Modifications in project or dependencies?
Project | |
---|---|
CCCL Infrastructure | |
libcu++ | |
+/- | CUB |
+/- | Thrust |
CUDA Experimental | |
+/- | pycuda |
+/- | CUDA C Core Library |
🏃 Runner counts (total jobs: 208)
# | Runner |
---|---|
171 | linux-amd64-cpu16 |
16 | linux-arm64-cpu16 |
12 | linux-amd64-gpu-v100-latest-1 |
9 | windows-amd64-cpu16 |
gevtushenko
approved these changes
Oct 8, 2024
🟩 CI finished in 1h 39m: Pass: 100%/208 | Total: 5d 17h | Avg: 39m 36s | Max: 1h 13m | Hits: 65%/16011
|
Project | |
---|---|
CCCL Infrastructure | |
libcu++ | |
+/- | CUB |
+/- | Thrust |
CUDA Experimental | |
pycuda | |
CUDA C Core Library |
Modifications in project or dependencies?
Project | |
---|---|
CCCL Infrastructure | |
libcu++ | |
+/- | CUB |
+/- | Thrust |
CUDA Experimental | |
+/- | pycuda |
+/- | CUDA C Core Library |
🏃 Runner counts (total jobs: 208)
# | Runner |
---|---|
171 | linux-amd64-cpu16 |
16 | linux-arm64-cpu16 |
12 | linux-amd64-gpu-v100-latest-1 |
9 | windows-amd64-cpu16 |
gevtushenko
approved these changes
Oct 8, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Closes #2458
ScanByKey
used to have the tile state comprising (1) the accumulated value and (2) theOffsetT
. TheOffsetT
part is used byReduceByKey
to figure the offsets of the reduced aggregate (i.e., the number of unique keys preceding the input so far). However, forScanByKey
this information is irrelevant, since we write exactly one partial sum per input item. The only remaining use for theOffsetT
in the case ofScanByKey
is to carry information on where a new segment begins (i.e., the head flags). I.e., to implement something like this:This PR got rid of having the key of the tile state be of
OffsetT
and instead moved to usingint
.Performance results
Note, we're planning on using
choose_offset_t
in theDeviceScan::*ByKey
interface, sou32
andu64
are the columns of interest.Summary: Comparing performance of different offset types, against previous main with offset type i32 PR:
Detailed H100 exclusive.by_key results, all offset types versus main
Summary: Comparing performance of different offset types, all after the changes in this PR:
Detailed H100 exclusive.by_key results, absolute of all offset types after this PR
Checklist