Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is the ExecRepoBench dataset file missing? #346

Open
kartikzheng opened this issue Dec 31, 2024 · 2 comments
Open

Is the ExecRepoBench dataset file missing? #346

kartikzheng opened this issue Dec 31, 2024 · 2 comments

Comments

@kartikzheng
Copy link

Where can I get the test.jsonl file mentioned in the ExecRepoBench dataset code? I couldn't find it on Huggingface. thanks.
https://huggingface.co/datasets/CSJianYang/ExecRepoBench

def evaluate_all_correctness(workers = 1, chunk_size = 10, conda_dir = "", env_dir = "", repo_root_path = "", output_path = ""): print("Path(__file__).resolve().parent:", Path(__file__).resolve().parent) objs = utils.read_jsonl_file(f"{Path(__file__).resolve().parent}/test_set/test.jsonl")

@cyente
Copy link
Collaborator

cyente commented Jan 3, 2025

i can simply click your url to find :), is there any problem with your internet?

@ytxmobile98
Copy link

Looks like you need to download the exec_repo_bench.jsonl file, and rename it to test_set/test.jsonl.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants