Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

functions for MVBench eval in demo_mistral.ipynb #240

Open
BennoKrojer opened this issue Sep 18, 2024 · 1 comment
Open

functions for MVBench eval in demo_mistral.ipynb #240

BennoKrojer opened this issue Sep 18, 2024 · 1 comment

Comments

@BennoKrojer
Copy link

Hello,

I am currently studying lots of things on MVBench and was wondering why you provided functions for EgoSchema such as check_answer_egoschema() but not for MVBench? Since MVBench is the main benchmark you released, it would be very helpful to also have those functions for MVBench to reproduce the numbers from your paper.
Do you have code for reproducing this as well?

Thank you!
Best,
Benno

@yinanhe
Copy link
Member

yinanhe commented Oct 11, 2024

I apologize for the late response. For information reproduce MVBench, you can check this notebook: MVBench Notebook.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants