Skip to content

Add an AI triage assistant #29646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 99 commits into from
May 29, 2025
Merged

Add an AI triage assistant #29646

merged 99 commits into from
May 29, 2025

Conversation

mattleibow
Copy link
Member

@mattleibow mattleibow commented May 23, 2025

This pull request introduces a set of GitHub Actions and supporting scripts to automate issue triage using AI. It provides functionality to generate prompts, run AI inference, merge results, and apply labels and comments to issues. Key changes include the addition of new composite actions, PowerShell scripts, and structured prompt templates for different triage scenarios.

New GitHub Actions and Scripts

Triage Prompt Generation:

  • Added triage-ai-gen-prompt action to generate prompt files from templates with support for variable replacement and label prefixing. (.github/actions/triage-ai-gen-prompt/action.yml, .github/actions/triage-ai-gen-prompt/process-prompt-template.ps1) [1] [2]

AI Inference:

  • Added triage-ai action to run AI inference using a specified model and endpoint, with inputs for prompt files and outputs for response files. (.github/actions/triage-ai/action.yml)

Label Merging and Application:

  • Added triage-apply action to merge label JSON files, summarize results, comment on issues, and apply labels. Includes steps for prompt generation and AI inference. (.github/actions/triage-apply/action.yml, .github/actions/triage-apply/merge-responses.ps1) [1] [2]

Flexible Labeling:

  • Added triage-labels action to process issues and apply labels based on different triage modes (e.g., multi-label, single-label, regression, missing-info). Supports dynamic prompt selection. (.github/actions/triage-labels/action.yml)

Prompt Templates

  • Added structured templates for system and user prompts, tailored to specific triage scenarios (e.g., missing information, multi-label assignment). (.github/actions/triage-apply/system-prompt.md, .github/actions/triage-apply/user-prompt.md, .github/actions/triage-labels/system-prompt-missinginfo.md, .github/actions/triage-labels/system-prompt-multilabel.md) [1] [2] [3] [4]

@mattleibow mattleibow closed this May 23, 2025
@mattleibow mattleibow reopened this May 23, 2025
@mattleibow mattleibow closed this May 23, 2025
@mattleibow mattleibow reopened this May 23, 2025
You are an expert triage assistant who is able to correctly
and accurately assign labels to new issues that are opened.

**Triage Process**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I would include more context like:

  • Focus on technical keywords, error messages, and platform-specific mentions.
  • Look for explicit platform mentions (iOS, Android, Windows, macOS, etc.).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Just trying to get this machine to do all the things and then, yeah, we need to iterate on the actual words.

@mattleibow mattleibow marked this pull request as ready for review May 28, 2025 22:41
@mattleibow mattleibow requested a review from a team as a code owner May 28, 2025 22:41
@mattleibow mattleibow changed the title Add some prompts Add an AI triage assistant May 29, 2025
@mattleibow mattleibow merged commit 4374621 into main May 29, 2025
4 checks passed
@mattleibow mattleibow deleted the dev/ai-labels branch May 29, 2025 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants