Skip to content

Allow definition of Model Settings for LLMJudge #1624

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
warp10-simonezambonim opened this issue Apr 30, 2025 · 3 comments · Fixed by #1662
Closed

Allow definition of Model Settings for LLMJudge #1624

warp10-simonezambonim opened this issue Apr 30, 2025 · 3 comments · Fixed by #1662
Assignees
Labels
evals Feature request New feature request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@warp10-simonezambonim
Copy link

Description

Although it is possible to choose the model, it is not possible to choose specific Model Settings for the LLM Judge;

Image

References

No response

@Kludex
Copy link
Member

Kludex commented May 1, 2025

@dmontagu Do you have thoughts, or can we add the parameter?

@Kludex Kludex added the Feature request New feature request label May 1, 2025
@dmontagu
Copy link
Contributor

dmontagu commented May 1, 2025

I think it's fine to add the parameter, just need to make sure it allows extra fields for the sake of supporting extra settings from vendor-specific subclasses of ModelSettings.

@Kludex Kludex added good first issue Good for newcomers help wanted Extra attention is needed labels May 6, 2025
@assadyousuf
Copy link
Contributor

assadyousuf commented May 8, 2025

@Kludex @dmontagu Put up PR with a fix for this: #1662. Lmk what you guys think

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
evals Feature request New feature request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants