Skip to content

Use actual PR summaries instead of bors placeholders when listing them #379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

moxian
Copy link

@moxian moxian commented May 3, 2025

When bisecting issues it often happens that a regression was introduced long enough ago that CI artifacts are unavailable, and we only have a list of PRs between two nightlies to go off of.

The current output only presents a numeric(!) list of PRs, which makes it very cumbersome to inspect those and narrow down. Best way I found was to handle this was to start a new comment on the issue, paste cargo-bisect-rustc output, then go to preview tab, and open all of the hotlinked issue numbers in new tabs.

This can be improved.
We can parse the bors autogenerated commit description and extract the original PR summary, and use that instead of just a number.
We can go even further and explicitly list all of the individual rolled-up PRs in case of rollups.

Output on current master (for mcve in rust-lang/rust#139089 as a random example):

> cargo bisect-rustc --start=2024-02-08 --end=2024-02-09 --regress=ice --preserve
checking the start range to find a passing nightly
installing nightly-2024-02-08
testing...
RESULT: nightly-2024-02-08, ===> Did not ICE

checking the end range to verify it does not pass
installing nightly-2024-02-09
testing...
RESULT: nightly-2024-02-09, ===> Found ICE

searched toolchains nightly-2024-02-08 through nightly-2024-02-09
checking last toolchain to determine final result
installing nightly-2024-02-09
testing...


********************************************************************************
Regression in nightly-2024-02-09
********************************************************************************

fetching https://static.rust-lang.org/dist/2024-02-08/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2024-02-08: 40 B / 40 B [======================================================] 100.00 % 123.73 KB/s converted 2024-02-08 to 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6
fetching https://static.rust-lang.org/dist/2024-02-09/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2024-02-09: 40 B / 40 B [======================================================] 100.00 % 155.07 KB/s converted 2024-02-09 to 98aa3624be70462d6a25ed5544333e3df62f4c66
looking for regression commit between 2024-02-08 and 2024-02-09
fetching (via remote github) commits from max(8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6, 2024-02-06) to 98aa3624be70462d6a25ed5544333e3df62f4c66
ending github query because we found starting sha: 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6
get_commits_between returning commits, len: 9
  commit[0] 2024-02-07: Auto merge of #120748 - Nadrieril:rollup-dj0qwv5, r=Nadrieril
  commit[1] 2024-02-08: Auto merge of #120381 - fee1-dead-contrib:reconstify-add, r=compiler-errors
  commit[2] 2024-02-08: Auto merge of #120521 - reitermarkus:generic-nonzero-constructors, r=dtolnay
  commit[3] 2024-02-08: Auto merge of #120558 - oli-obk:missing_impl_item_ice, r=estebank
  commit[4] 2024-02-08: Auto merge of #120579 - GuillaumeGomez:prevent-running-unneeded-code, r=notriddle
  commit[5] 2024-02-08: Auto merge of #120550 - oli-obk:track_errors8, r=estebank
  commit[6] 2024-02-08: Auto merge of #120767 - matthiaskrgr:rollup-0k8ib1c, r=matthiaskrgr
  commit[7] 2024-02-08: Auto merge of #120544 - BoxyUwU:enter_forall, r=lcnr
  commit[8] 2024-02-08: Auto merge of #120807 - matthiaskrgr:rollup-1pf3glu, r=matthiaskrgr
ERROR: no CI builds available between 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6 and 98aa3624be70462d6a25ed5544333e3df62f4c66 within last 167 days

Output with the PR:

> cargo bisect-rustc --start=2024-02-08 --end=2024-02-09 --regress=ice --preserve
checking the start range to find a passing nightly
installing nightly-2024-02-08
testing...
RESULT: nightly-2024-02-08, ===> Did not ICE

checking the end range to verify it does not pass
installing nightly-2024-02-09
testing...
RESULT: nightly-2024-02-09, ===> Found ICE

searched toolchains nightly-2024-02-08 through nightly-2024-02-09
checking last toolchain to determine final result
installing nightly-2024-02-09
testing...


********************************************************************************
Regression in nightly-2024-02-09
********************************************************************************

fetching https://static.rust-lang.org/dist/2024-02-08/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2024-02-08: 40 B / 40 B [=======================================================] 100.00 % 76.20 KB/s converted 2024-02-08 to 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6
fetching https://static.rust-lang.org/dist/2024-02-09/channel-rust-nightly-git-commit-hash.txt
nightly manifest 2024-02-09: 40 B / 40 B [=======================================================] 100.00 % 96.31 KB/s converted 2024-02-09 to 98aa3624be70462d6a25ed5544333e3df62f4c66
looking for regression commit between 2024-02-08 and 2024-02-09
fetching (via remote github) commits from max(8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6, 2024-02-06) to 98aa3624be70462d6a25ed5544333e3df62f4c66
ending github query because we found starting sha: 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6
PRs in range:
  - #120748 (Rollup of 13 pull requests) by Nadrieril
    - #110482 (Add armv8r-none-eabihf target for the Cortex-R52.)
    - #119162 (Add unstable `-Z direct-access-external-data` cmdline flag for `rustc`)
    - #120302 (various const interning cleanups)
    - #120455 ( Add FileCheck annotations to MIR-opt SROA tests)
    - #120470 (Mark "unused binding" suggestion as maybe incorrect)
    - #120479 (Suggest turning `if let` into irrefutable `let` if appropriate)
    - #120564 (coverage: Split out counter increment sites from BCB node/edge counters)
    - #120633 (pattern_analysis: gather up place-relevant info)
    - #120664 (Add parallel rustc ui tests)
    - #120726 (Don't use bashism in checktools.sh)
    - #120733 (MirPass: make name more const)
    - #120735 (Remove some `unchecked_claim_error_was_emitted` calls)
    - #120746 (Record coroutine kind in coroutine generics)
  - #120381 (Reconstify `Add`) by fee1-dead-contrib
  - #120521 (Make `NonZero` constructors generic.) by reitermarkus
  - #120558 (Stop bailing out from compilation just because there were incoherent traits) by oli-obk
  - #120579 (Prevent running some code if it is already in the map) by GuillaumeGomez
  - #120550 (Continue to borrowck even if there were previous errors) by oli-obk
  - #120767 (Rollup of 9 pull requests) by matthiaskrgr
    - #119592 (resolve: Unload speculatively resolved crates before freezing cstore)
    - #120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
    - #120206 (hir: Make sure all `HirId`s have corresponding HIR `Node`s)
    - #120214 (match lowering: consistently lower bindings deepest-first)
    - #120688 (GVN: also turn moves into copies with projections)
    - #120702 (docs: also check the inline stmt during redundant link check)
    - #120727 (exhaustiveness: Prefer "`0..MAX` not covered" to "`_` not covered")
    - #120734 (Add `SubdiagnosticMessageOp` as a trait alias.)
    - #120739 (improve pretty printing for associated items in trait objects)
  - #120544 (Introduce `enter_forall` to supercede `instantiate_binder_with_placeholders`) by BoxyUwU
  - #120807 (Rollup of 9 pull requests) by matthiaskrgr
    - #120590 (Remove unused args from functions)
    - #120750 (No need to take `ImplTraitContext` by ref)
    - #120769 (make future diffs minimal)
    - #120772 (Remove myself from review rotation.)
    - #120775 (Make `min_exhaustive_patterns` match `exhaustive_patterns` better)
    - #120778 (Deduplicate `tcx.instance_mir(instance)` calls in `try_instance_mir`)
    - #120782 (Fix mir pass ICE in the presence of other errors)
    - #120783 (Add release note for new ambiguous_wide_pointer_comparisons lint)
    - #120801 (Avoid ICE in drop recursion check in case of invalid drop impls)
ERROR: no CI builds available between 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6 and 98aa3624be70462d6a25ed5544333e3df62f4c66 within last 167 days

Even with the change in place, one would still probably want to paste the output into the github commend field and then look at the Preview to click through the issues, but at least this allows to filter out the obviously irrelevant ones easier.

Also expand rollup description to show individual rolled up prs
@apiraino
Copy link
Contributor

apiraino commented May 5, 2025

This is an interesting addition. Just like you I also always inspect the rollup content manually.

Just one thought. The new output is now (understandably) verbose. Do you think it can be compressed with some clever markdown tricks?

Here an example, it's half-baked and does not render great (the Markdown nested syntax is very sensitive to spaces):

PRs in range:

  • #120748 (Rollup of 13 pull requests) by Nadrieril

    PRs in rollup

    • #110482 (Add armv8r-none-eabihf target for the Cortex-R52.)
    • #119162 (Add unstable -Z direct-access-external-data cmdline flag for rustc)
    • #120302 (various const interning cleanups)
    • #120455 ( Add FileCheck annotations to MIR-opt SROA tests)
    • #120470 (Mark "unused binding" suggestion as maybe incorrect)
    • #120479 (Suggest turning if let into irrefutable let if appropriate)
    • #120564 (coverage: Split out counter increment sites from BCB node/edge counters)
    • #120633 (pattern_analysis: gather up place-relevant info)
    • #120664 (Add parallel rustc ui tests)
    • #120726 (Don't use bashism in checktools.sh)
    • #120733 (MirPass: make name more const)
    • #120735 (Remove some unchecked_claim_error_was_emitted calls)
    • #120746 (Record coroutine kind in coroutine generics)

  • #120381 (Reconstify Add) by fee1-dead-contrib

  • #120521 (Make NonZero constructors generic.) by reitermarkus

  • #120558 (Stop bailing out from compilation just because there were incoherent traits) by oli-obk

  • #120579 (Prevent running some code if it is already in the map) by GuillaumeGomez

  • #120550 (Continue to borrowck even if there were previous errors) by oli-obk

  • #120767 (Rollup of 9 pull requests) by matthiaskrgr

    PRs in rollup

    • #119592 (resolve: Unload speculatively resolved crates before freezing cstore)
    • #120103 (Make it so that async-fn-in-trait is compatible with a concrete future in implementation)
    • #120206 (hir: Make sure all HirIds have corresponding HIR Nodes)
    • #120214 (match lowering: consistently lower bindings deepest-first)
    • #120688 (GVN: also turn moves into copies with projections)
    • #120702 (docs: also check the inline stmt during redundant link check)
    • #120727 (exhaustiveness: Prefer "0..MAX not covered" to "_ not covered")
    • #120734 (Add SubdiagnosticMessageOp as a trait alias.)
    • #120739 (improve pretty printing for associated items in trait objects)

  • #120544 (Introduce enter_forall to supercede instantiate_binder_with_placeholders) by BoxyUwU

  • #120807 (Rollup of 9 pull requests) by matthiaskrgr

    PRs in rollup

    • #120590 (Remove unused args from functions)
    • #120750 (No need to take ImplTraitContext by ref)
    • #120769 (make future diffs minimal)
    • #120772 (Remove myself from review rotation.)
    • #120775 (Make min_exhaustive_patterns match exhaustive_patterns better)
    • #120778 (Deduplicate tcx.instance_mir(instance) calls in try_instance_mir)
    • #120782 (Fix mir pass ICE in the presence of other errors)
    • #120783 (Add release note for new ambiguous_wide_pointer_comparisons lint)
    • #120801 (Avoid ICE in drop recursion check in case of invalid drop impls)

ERROR: no CI builds available between 8ace7ea1f7cbba7b4f031e66c54ca237a0d65de6 and 98aa3624be70462d6a25ed5544333e3df62f4c66 within last 167 days

@moxian
Copy link
Author

moxian commented May 5, 2025

No, I don't think so. Three reasons:

  1. rolled up PRs are not any less important or relevant when bisecting, so hiding those (and not others) feels odd and arbitrary
  2. for very silly reasons I feel bad about "needlessly tagging innocent PRs", so when leaving a comment on the issue, I tend to post the bisection results inside a code block to prevent hotlinking. Like I did in the OP here. That would look bad with collapsibles. (But I can accept that this is a me issue and should be dismissed)
  3. Similarly, this would also look bad in console (but we're already outputting markdown in console in other cases)

If PR list looks to long, then it's better to hide the entirety of it (and but just parts) under collapsible, IMO. I wouldn't mind adding those collapsible decorations for the entire list in the output by default, if we think it's a good idea.

@apiraino
Copy link
Contributor

apiraino commented May 5, 2025

I don't feel too strongly about my suggestion but I think some form of collapsible could help reducing the clutter for visitors not interested in the bisection.

I also totally agree this bisection blurb should avoid tagging PRs in general, that's needless background noise across the repository. I use a github trick to avoid backlinks:

[https://togithub.com/rust/pull/123456](#123456)

Note the "togithub.com" domain. Could it be a solution to post backlinks quietly?

In any case thanks for submitting your idea (I like it!)

@moxian moxian marked this pull request as draft May 6, 2025 04:48
src/main.rs Outdated
Comment on lines 983 to 985
let bors_re = regex::Regex::new(
r"Auto merge of #(?<pr_num>\d+) - (?<author>[^:]+):.*\n\s*\n(?<pr_summary>.*)",
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this function is called in a loop, regexp instantiation can stay out of the loop for perf. reasons (also the other one down in this function)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, yeah, thanks for cathing this one!
For some reason I was absolutely convinced that regex instantiation is internally cached, and re-creating one in a loop is effectively free. But now that I check that, the docs explicitly say that this is not the case.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I gave it another thought and decided against it for now.
This is only called like 10 times tops in a normal run (once per PR in a nightly). Removing the regex re-parsing makes the code less pretty, and if perf is a concern, then there are much lower hanging fruit. Namely we create an entire reqwest::blocking::Client (and associated tokio runtime) to fetch the commits descriptions, and I'm confident that it takes way more time than a regex.

Still, I'm address the extra regex copies if desired, but I wanted to push back a little bit for this round.

@moxian
Copy link
Author

moxian commented May 9, 2025

Sorry for having it sitting for a while, this is a bit stale but absolutely not abandoned!
I'm being distracted by other shinies (both IRL and in hobbyland), but I will experiement with collapsibles presenatation, figure out what looks better and push the update in the next couple of days (although I cannot guarantee this will get resolved before monday).

@apiraino
Copy link
Contributor

apiraino commented May 9, 2025

no worries @moxian take your time :)

@moxian
Copy link
Author

moxian commented May 12, 2025

Added the collapsible + code block. Looks like this now:

Regression in nightly-2024-11-20. PRs in range:
 - #133179 (Rollup of 5 pull requests) by GuillaumeGomez
   - #133156 (typo in config.example.toml)
   - #133157 (stability: remove skip_stability_check_due_to_privacy)
   - #133163 (remove pointless cold_path impl in interpreter)
   - #133169 (Update autolabels for T-compiler and T-bootstrap)
   - #133171 (Add the missing quotation mark in comment)
 - #132460 (Use `TypingMode` throughout the compiler instead of `ParamEnv`) by lcnr
 - #124780 (Improve VecCache under parallel frontend) by Mark-Simulacrum
 - #133193 (Rollup of 9 pull requests) by fmease
   - #132758 (Improve `{BTreeMap,HashMap}::get_key_value` docs.)
   - #133180 ([rustdoc] Fix items with generics not having their jump to def link generated)
   - #133181 (Update books)
   - #133182 (const_panic: inline in bootstrap builds to avoid f16/f128 crashes)
   - #133185 (rustdoc-search: use smart binary search in bitmaps)
   - #133186 (Document s390x-unknown-linux targets)
   - #133187 (Add reference annotations for diagnostic attributes)
   - #133191 (rustdoc book: Move `--test-builder(--wrapper)?` docs to unstable section.)
   - #133192 (RELEASES.md: Don't document unstable `--test-build-wrapper`)
 - #132623 (`rustc_borrowck` cleanups, part 2) by nnethercote
 - #133164 (interpret: do not ICE when a promoted fails with OOM) by RalfJung
 - #133205 (Rollup of 4 pull requests) by matthiaskrgr
   - #131081 (Use `ConstArgKind::Path` for all single-segment paths, not just params under `min_generic_const_args`)
   - #132577 (Report the `unexpected_cfgs` lint in external macros)
   - #133023 (Merge `-Zhir-stats` into `-Zinput-stats`)
   - #133200 (ignore an occasionally-failing test in Miri)
 - #132761 (Resolve tweaks) by nnethercote

I struggle to show how it would look in console because codeblock-inside-codeblock no worky, but I'll say that I'm happy with the presentation 🙃

There is maybe an opportunity to add extra syntax highlight, but i'm undecided on whether this is an improvement and what language to use, if any.

examples
rust:
 - #133179 (Rollup of 5 pull requests) by GuillaumeGomez
   - #133156 (typo in config.example.toml)
   - #133157 (stability: remove skip_stability_check_due_to_privacy)
   - #133163 (remove pointless cold_path impl in interpreter)
   - #133169 (Update autolabels for T-compiler and T-bootstrap)
   - #133171 (Add the missing quotation mark in comment)
 - #132460 (Use `TypingMode` throughout the compiler instead of `ParamEnv`) by lcnr
 - #124780 (Improve VecCache under parallel frontend) by Mark-Simulacrum
 - #133193 (Rollup of 9 pull requests) by fmease
   - #132758 (Improve `{BTreeMap,HashMap}::get_key_value` docs.)
   - #133180 ([rustdoc] Fix items with generics not having their jump to def link generated)
   - #133181 (Update books)
   - #133182 (const_panic: inline in bootstrap builds to avoid f16/f128 crashes)
   - #133185 (rustdoc-search: use smart binary search in bitmaps)
   - #133186 (Document s390x-unknown-linux targets)
   - #133187 (Add reference annotations for diagnostic attributes)
   - #133191 (rustdoc book: Move `--test-builder(--wrapper)?` docs to unstable section.)
   - #133192 (RELEASES.md: Don't document unstable `--test-build-wrapper`)
 - #132623 (`rustc_borrowck` cleanups, part 2) by nnethercote
 - #133164 (interpret: do not ICE when a promoted fails with OOM) by RalfJung
 - #133205 (Rollup of 4 pull requests) by matthiaskrgr
   - #131081 (Use `ConstArgKind::Path` for all single-segment paths, not just params under `min_generic_const_args`)
   - #132577 (Report the `unexpected_cfgs` lint in external macros)
   - #133023 (Merge `-Zhir-stats` into `-Zinput-stats`)
   - #133200 (ignore an occasionally-failing test in Miri)
 - #132761 (Resolve tweaks) by nnethercote
markdown
 - #133179 (Rollup of 5 pull requests) by GuillaumeGomez
   - #133156 (typo in config.example.toml)
   - #133157 (stability: remove skip_stability_check_due_to_privacy)
   - #133163 (remove pointless cold_path impl in interpreter)
   - #133169 (Update autolabels for T-compiler and T-bootstrap)
   - #133171 (Add the missing quotation mark in comment)
 - #132460 (Use `TypingMode` throughout the compiler instead of `ParamEnv`) by lcnr
 - #124780 (Improve VecCache under parallel frontend) by Mark-Simulacrum
 - #133193 (Rollup of 9 pull requests) by fmease
   - #132758 (Improve `{BTreeMap,HashMap}::get_key_value` docs.)
   - #133180 ([rustdoc] Fix items with generics not having their jump to def link generated)
   - #133181 (Update books)
   - #133182 (const_panic: inline in bootstrap builds to avoid f16/f128 crashes)
   - #133185 (rustdoc-search: use smart binary search in bitmaps)
   - #133186 (Document s390x-unknown-linux targets)
   - #133187 (Add reference annotations for diagnostic attributes)
   - #133191 (rustdoc book: Move `--test-builder(--wrapper)?` docs to unstable section.)
   - #133192 (RELEASES.md: Don't document unstable `--test-build-wrapper`)
 - #132623 (`rustc_borrowck` cleanups, part 2) by nnethercote
 - #133164 (interpret: do not ICE when a promoted fails with OOM) by RalfJung
 - #133205 (Rollup of 4 pull requests) by matthiaskrgr
   - #131081 (Use `ConstArgKind::Path` for all single-segment paths, not just params under `min_generic_const_args`)
   - #132577 (Report the `unexpected_cfgs` lint in external macros)
   - #133023 (Merge `-Zhir-stats` into `-Zinput-stats`)
   - #133200 (ignore an occasionally-failing test in Miri)
 - #132761 (Resolve tweaks) by nnethercote

@moxian moxian marked this pull request as ready for review May 12, 2025 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants