Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] write_operations.csv does not differentiate between CPU and GPU operations #1564

Open
sayedbilalbari opened this issue Feb 27, 2025 · 0 comments
Open

[BUG] write_operations.csv does not differentiate between CPU and GPU operations #1564

sayedbilalbari opened this issue Feb 27, 2025 · 0 comments
Labels
API change A changeA change affecting the output (add/remove/rename files, add/remove/rename columns) bug Something isn't working core_tools Scope the core module (scala)

Comments

@sayedbilalbari
Copy link
Collaborator

Currently the write operations file generated, parses HadoopFsInput command and figures out the exec commands.
Ideally, this should differentiate between CPU and GPU writes and dump the correct exec operation.

spark-rapids-tools/core/src/main/scala/com/nvidia/spark/rapids/tool/planparser/DataWritingCommandExecParser.scala

Line 364 in be68be4

val cmdWrapper = if (DeltaLakeHelper.acceptsWriteOp(node)) {

@sayedbilalbari sayedbilalbari added ? - Needs Triage API change A changeA change affecting the output (add/remove/rename files, add/remove/rename columns) bug Something isn't working core_tools Scope the core module (scala) labels Feb 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
API change A changeA change affecting the output (add/remove/rename files, add/remove/rename columns) bug Something isn't working core_tools Scope the core module (scala)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mattahrens @sayedbilalbari