Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support model evaluation #197

Merged
merged 15 commits into from
Dec 6, 2024
Merged

Support model evaluation #197

merged 15 commits into from
Dec 6, 2024

Conversation

ganisback
Copy link
Collaborator

@ganisback ganisback commented Dec 5, 2024

for LLM evaluation, provide one-stop platform for large model evaluation, aiming to provide a fair, open, and reproducible benchmark for large model evaluation.
currently supported evaluation

  1. opencompass https://github.com/open-compass/opencompass
  2. llm evaluation harness: https://github.com/EleutherAI/lm-evaluation-harness

Yiling-J and others added 6 commits December 5, 2024 22:21
Support argo workflow system

Support evaluation datasets

support opencompass
support harness evaluation
@starship-github
Copy link

🚀 The StarShip CodeReviewer has been triggered with action(s): review, evaluate, describe, linter.

@ganisback ganisback changed the title Support evaluation Support model evaluation Dec 6, 2024
@ganisback ganisback merged commit 5d7b63c into main Dec 6, 2024
4 checks passed
@ganisback ganisback deleted the support-evaluation branch December 6, 2024 04:34
@starship-github
Copy link

The StarShip CodeReviewer was triggered but terminated because it encountered an issue: The MR state is not opened.

Tips

CodeReview Commands (invoked as MR or PR comments)

  • @codegpt /review to trigger an code review.
  • @codegpt /evaluate to trigger code evaluation process.
  • @codegpt /describe to regenerate the summary of the MR.
  • @codegpt /secscan to scan security vulnerabilities for the MR or the Repository.
  • @codegpt /help to get help.

CodeReview Discussion Chat

There are 2 ways to chat with Starship CodeReview:

  • Review comments: Directly reply to a review comment made by StarShip.
    Example:
    • @codegpt How to fix this bug?
  • Files and specific lines of code (under the "Files changed" tab):
    Tag @codegpt in a new review comment at the desired location with your query.
    Examples:
    • @codegpt generate unit testing code for this code snippet.

Note: Be mindful of the bot's finite context window.
It's strongly recommended to break down tasks such as reading entire modules into smaller chunks.
For a focused discussion, use review comments to chat about specific files and their changes, instead of using the MR/PR comments.

CodeReview Documentation and Community

  • Visit our Documentation
    for detailed information on how to use Starship CodeReview.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants