-
Notifications
You must be signed in to change notification settings - Fork 12.9k
Fix Copilot instructions to emphasize mandatory lint, test, and format steps #61950
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…tical build commands Co-authored-by: RyanCavanaugh <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I admit that I was unclear. When the agent clones the repo, it can assume that lint, tests, and formatting are clean, since all PRs have to pass those gates before merging. There's no need to lint, test, or format on a fresh clone.
It's only important to run any of these steps after it makes changes. The agent should figure out how to make a failing testcase, root-cause the issue, fix the bug, verify the test passes, then run all the tests, assess what to do with any failures, repeat, then run lint, fix any lint issues, re-run the tests if needed, then format as a very last step.
…/format after changes Co-authored-by: RyanCavanaugh <[email protected]>
Updated the instructions to clarify that lint/test/format steps should only run AFTER making changes, not before. Removed the confusing "MANDATORY STEPS" section and repositioned guidance in the workflow. Agents can now assume fresh clones are clean. Fixed in commit 82a1d57. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem is that the agent is failing to run these steps, not that's running them too often on a fresh clone. Emphasize that before it considers wrapping up and pushing a commit for PR, it needs to have run the tests, format, and linting. We can't spend any time looking at PRs that fail those steps.
…dress maintainer feedback about agents skipping critical verification steps Co-authored-by: RyanCavanaugh <[email protected]>
Added prominent "🚨 CRITICAL: Before Finishing Your Work 🚨" section emphasizing agents MUST run runtests-parallel, lint, and format before committing. Strengthened workflow language with "MANDATORY" and "REQUIRED" warnings. Fixed in commit 645efcb. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might work!
The existing Copilot instructions were not sufficiently emphasizing critical mandatory steps, leading to Copilot frequently skipping essential build verification steps. This was causing maintainer frustration as described in the issue.
Changes Made
1. Added Prominent Mandatory Steps Section
Added a highly visible "⚠️ MANDATORY STEPS - DO NOT SKIP! ⚠️ " section at the top of
.github/copilot-instructions.md
with clear 3-step process:npx hereby lint
- Run linting FIRST before making any changesnpx hereby runtests-parallel
- Required despite 15-minute runtimenpx hereby format
- Always run as the very last step2. Strengthened Language Throughout Document
3. Updated Build Task Comments
Enhanced the common build tasks section to clearly mark the three critical commands:
4. Reinforced Workflow Section
Updated the "Recommended Workflow" section to use stronger mandatory language:
npx hereby runtests-parallel
" with patience remindernpx hereby lint
and fix ALL issues"Verification
All mandatory steps were tested and confirmed working:
npx hereby lint
(passed in 39.4s)npx hereby runtests-parallel
(98,776 tests passed in 12m)npx hereby format
(completed in 97ms)These changes ensure Copilot cannot miss the critical build verification steps that are essential for maintaining code quality and avoiding maintainer frustration.
Fixes #61949.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.