Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(reports): removing index column #32366

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

SkinnyPigeon
Copy link
Contributor

SUMMARY

This supersedes this PR: 29335
By default, the URL generated for CSVs includes the argument type=post_processed. This can lead to an index column being included. This is often an undesirable inclusion and our analytics team has repeatedly asked for it to be removed. This PR adds a checkbox to the AlertReportModal that allows users to manually add it.

I've defaulted it to be disabled which may not be what users are after. I am happy to reverse it so that it replicates the current behaviour by default.

Massive thanks to Colin for opening the PR on my fork to add the modal.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

Sorry, I couldn't get reports to send with my local setup. However, as well as the unit tests, I added logging to confirm that the URL was indeed being mutated to remove the type=post_processed argument from the request

ADDITIONAL INFORMATION

  • Has associated issue: FIxes 22981
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@github-actions github-actions bot added risk:db-migration PRs that require a DB migration api Related to the REST API labels Feb 24, 2025
@dosubot dosubot bot added the alert-reports Namespace | Anything related to the Alert & Reports feature label Feb 24, 2025
@SkinnyPigeon SkinnyPigeon changed the title feat(reports) removing index column feat (reports) removing index column Feb 24, 2025
Copy link

codecov bot commented Feb 24, 2025

Codecov Report

Attention: Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review.

Project coverage is 83.47%. Comparing base (76d897e) to head (29d18d4).
Report is 1495 commits behind head on master.

Files with missing lines Patch % Lines
superset/commands/report/execute.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master   #32366       +/-   ##
===========================================
+ Coverage   60.48%   83.47%   +22.98%     
===========================================
  Files        1931      547     -1384     
  Lines       76236    39106    -37130     
  Branches     8568        0     -8568     
===========================================
- Hits        46114    32643    -13471     
+ Misses      28017     6463    -21554     
+ Partials     2105        0     -2105     
Flag Coverage Δ
hive 48.51% <33.33%> (-0.65%) ⬇️
javascript ?
mysql 75.85% <40.00%> (?)
postgres 75.93% <40.00%> (?)
presto 53.04% <33.33%> (-0.76%) ⬇️
python 83.47% <93.33%> (+19.98%) ⬆️
sqlite 75.43% <40.00%> (?)
unit 61.12% <86.66%> (+3.50%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

@korbit-ai korbit-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review by Korbit AI

Korbit automatically attempts to detect when you fix issues in new commits.
Category Issue Fix Detected
Functionality Post-processing removal affects more than index column ▹ view
Files scanned
File Path Reviewed
superset/migrations/versions/2025-02-24_17-52_3b86f563edbc_add_include_index_to_report_schedule.py
superset/commands/report/utils.py
superset-frontend/src/features/alerts/types.ts
superset/reports/models.py
superset/reports/schemas.py
superset/reports/api.py
superset/commands/report/execute.py
superset-frontend/src/features/alerts/AlertReportModal.tsx

Explore our documentation to understand the languages and file types we support and the files we ignore.

Need a new review? Comment /korbit-review on this PR and I'll review your latest changes.

Korbit Guide: Usage and Customization

Interacting with Korbit

  • You can manually ask Korbit to review your PR using the /korbit-review command in a comment at the root of your PR.
  • You can ask Korbit to generate a new PR description using the /korbit-generate-pr-description command in any comment on your PR.
  • Too many Korbit comments? I can resolve all my comment threads if you use the /korbit-resolve command in any comment on your PR.
  • Chat with Korbit on issues we post by tagging @korbit-ai in your reply.
  • Help train Korbit to improve your reviews by giving a 👍 or 👎 on the comments Korbit posts.

Customizing Korbit

  • Check out our docs on how you can make Korbit work best for you and your team.
  • Customize Korbit for your organization through the Korbit Console.

Feedback and Support

Comment on lines 367 to 368
if not self._report_schedule.include_index:
url = remove_post_processed(url)

This comment was marked as resolved.

Comment on lines 367 to 368
if not self._report_schedule.include_index:
url = remove_post_processed(url)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding logging above and below this shows the URL being mutated when the include index option is left unchecked

@sadpandajoe
Copy link
Member

@SkinnyPigeon I'd change this to be the default behavior by default as that is how the current app behaves.

@sadpandajoe sadpandajoe changed the title feat (reports) removing index column feat(reports): removing index column Feb 25, 2025
@SkinnyPigeon
Copy link
Contributor Author

Thanks @sadpandajoe. I've no idea why I didn't think to do it that way in the first place. I've reversed the behaviour so the setting is not remove_index. This defaults to False so all existing reports should be unaffected but users can edit existing reports or create new ones and set the remove_index to True. If this check is positive, the query parameter URL is stripped of the type=post_processed element

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert-reports Namespace | Anything related to the Alert & Reports feature api Related to the REST API risk:db-migration PRs that require a DB migration size/L
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants