Skip to content

Issues: embeddings-benchmark/mteb

Merge v2.0.0: Overview issue
#1791 opened Jan 13, 2025 by KennethEnevoldsen
Open 2
Overview: Leaderboard release
#1867 opened Jan 24, 2025 by x-tabdeveloping
Open 26
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

173 Open 586 Closed
173 Open 586 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Add an overview benchmark enhancement New feature or request leaderboard issues related to the leaderboard new benchmark Issues related to adding a new benchmark
#2038 opened Feb 12, 2025 by KennethEnevoldsen
1
Dataset 'mteb/sts22-crosslingual-sts' doesn't exist on the Hub or cannot be accessed.
#2036 opened Feb 12, 2025 by omarnj-lab
1
Add leaderboard unit tests leaderboard issues related to the leaderboard
#2034 opened Feb 11, 2025 by Muennighoff
Freeze columns for better leaderboard UI good first issue Good for newcomers leaderboard issues related to the leaderboard
#2033 opened Feb 11, 2025 by Muennighoff
@shikhar1729
1
load_data() does not load self.dataset for all classes v2 Issues and PRs related to `v2` branch
#2030 opened Feb 11, 2025 by jhyuklee
2
Add MIEB as a benchmark mieb The image extension of MTEB
#2028 opened Feb 11, 2025 by isaac-chung
1
@isaac-chung
Add https://huggingface.co/codesage/codesage-large-v2 good first issue Good for newcomers new model Questions related to adding a new model to the benchmark
#2020 opened Feb 7, 2025 by Muennighoff
[v2] Refactior task specific scores v2 Issues and PRs related to `v2` branch
#2016 opened Feb 7, 2025 by Samoed v2.0.0
Validate language codes for ModelMeta Languages
#2013 opened Feb 7, 2025 by KennethEnevoldsen
CI: Seperate out tests that fail due to HTTP calls
#2012 opened Feb 7, 2025 by KennethEnevoldsen
1
4
Clicking model names does not open new window enhancement New feature or request leaderboard issues related to the leaderboard
#2005 opened Feb 7, 2025 by Muennighoff
6
Add GTR models good first issue Good for newcomers leaderboard issues related to the leaderboard new model Questions related to adding a new model to the benchmark
#2000 opened Feb 6, 2025 by Muennighoff
Add FRIDA models good first issue Good for newcomers new model Questions related to adding a new model to the benchmark
#1999 opened Feb 6, 2025 by Muennighoff
Add SGPT models good first issue Good for newcomers new model Questions related to adding a new model to the benchmark
#1997 opened Feb 6, 2025 by Muennighoff
Add instructor models good first issue Good for newcomers new model Questions related to adding a new model to the benchmark
#1996 opened Feb 6, 2025 by Muennighoff
Leaderboard: Gradio Sidebars for better UI enhancement New feature or request good first issue Good for newcomers leaderboard issues related to the leaderboard
#1988 opened Feb 5, 2025 by Muennighoff
2
Leaderboard: Add more explainers leaderboard issues related to the leaderboard
#1986 opened Feb 5, 2025 by Muennighoff
1
@KennethEnevoldsen
5
Leaderboard (CoIR): Missing results for COIRCodeSearchNetRetrieval causes mean to be blank enhancement New feature or request leaderboard issues related to the leaderboard
#1980 opened Feb 5, 2025 by Muennighoff
6
Leaderboard: BRIGHT Long gone leaderboard issues related to the leaderboard
#1978 opened Feb 5, 2025 by Muennighoff
1
@KennethEnevoldsen
Leaderboard: Many results on BRIGHT missing leaderboard issues related to the leaderboard
#1977 opened Feb 5, 2025 by Muennighoff
1
Leaderboard: OpenAI results gone? leaderboard issues related to the leaderboard
#1976 opened Feb 5, 2025 by Muennighoff
4
[v2] MIEB classes with duplicate names v2 Issues and PRs related to `v2` branch
#1974 opened Feb 5, 2025 by Samoed
2
Reduce load time for the leaderboard enhancement New feature or request leaderboard issues related to the leaderboard
#1972 opened Feb 5, 2025 by KennethEnevoldsen
2 tasks
7
Ensure that descriptive stats are computed for all tasks enhancement New feature or request
#1970 opened Feb 5, 2025 by KennethEnevoldsen
2 tasks
1
Make some dependencies optional
#1966 opened Feb 5, 2025 by x-tabdeveloping
4
ProTip! Adding no:label will show everything without a label.