Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Add SQL ID column in failed_jobs.csv in Profiling tools output #1563

Open
cindyyuanjiang opened this issue Feb 27, 2025 · 0 comments · May be fixed by #1567
Open

[FEA] Add SQL ID column in failed_jobs.csv in Profiling tools output #1563

cindyyuanjiang opened this issue Feb 27, 2025 · 0 comments · May be fixed by #1567
Assignees
@amahussein
Labels
API change A changeA change affecting the output (add/remove/rename files, add/remove/rename columns) core_tools Scope the core module (scala) feature request New feature or request

Comments

@cindyyuanjiang
Copy link
Collaborator

cindyyuanjiang commented Feb 27, 2025 •
edited
Loading

Is your feature request related to a problem? Please describe.
We wish to have SQL query level success/failure status from Profiling tool output. We can add a "SQL ID" column in failed_jobs.csv to include this info.
@amahussein amahussein added core_tools Scope the core module (scala) and removed ? - Needs Triage labels Feb 28, 2025
@amahussein amahussein self-assigned this Feb 28, 2025
@cindyyuanjiang cindyyuanjiang changed the title [FEA] Add failed_sql.csv to Profiling tools output [FEA] Add SQL ID column in failed_stages.csv in Profiling tools output Feb 28, 2025
@cindyyuanjiang cindyyuanjiang changed the title [FEA] Add SQL ID column in failed_stages.csv in Profiling tools output [FEA] Add SQL ID column in failed_jobs.csv in Profiling tools output Feb 28, 2025
amahussein added a commit to amahussein/spark-rapids-tools that referenced this issue Mar 3, 2025
@amahussein
Add sqlID column to failed_jobs.csv
a3d4853 
Signed-off-by: Ahmed Hussein (amahussein) <[email protected]>

Fixes NVIDIA#1563

- adds column `sqlID` to failed_jobs.csv
- the column might be empty if the job has no sqlID attached to it
This was referenced Mar 3, 2025
Open
Open
@amahussein amahussein added the API change A changeA change affecting the output (add/remove/rename files, add/remove/rename columns) label Mar 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amahussein amahussein

Labels
API change A changeA change affecting the output (add/remove/rename files, add/remove/rename columns) core_tools Scope the core module (scala) feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add sqlID column to failed_jobs.csv amahussein/spark-rapids-tools
2 participants
@cindyyuanjiang @amahussein