Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding dataset for new chimera frontend #112

Merged
merged 1 commit into from
Mar 31, 2022

Conversation

Ronan-Hix
Copy link
Contributor

Adds new datasets for use with the answer testing of the new Chimera frontend.

@yt-fido
Copy link
Collaborator

yt-fido commented Mar 25, 2022

Can one of the admins verify this patch?

@Xarthisius
Copy link
Member

Hi @Ronan-Hix ! Where's the data mentioned in this PR? It hasn't been uploaded to yt-project.org/data

@Xarthisius
Copy link
Member

I'm also looking at the sizes of those datasets and they're quite big. Would it be possible to find something smaller for tests?

@Ronan-Hix
Copy link
Contributor Author

Hey,

I was told to curl the data using curl -T [datafile name].tar.gz http://use.yt/upload/, if it should be uploaded to a different location, I can do that. As for data size, the F37 dataset could be decreased to <300MB without any issue. The C15 dataset is comprised of 45 "wedges" that combine to make up the simulation grid at a given timestep. One of the tests for the frontend checks that these wedges are meshed together correctly on read-in. I could decrease the size of the dataset by only including a few of the 45 wedges, so long as that wouldn't be considered to undermine the rigor of the test.

@Xarthisius
Copy link
Member

I was told to curl the data using curl -T [datafile name].tar.gz http://use.yt/upload/

That's fine, but you haven't shared the urls that use.yt/upload/ returned

@Ronan-Hix
Copy link
Contributor Author

Sorry, I'm new to the process. What steps should I take to share those urls? Apologies if there's a standard process for getting data moved from use.yt/upload/ to yt-project/data that I'm unaware of.

@Xarthisius
Copy link
Member

Sorry, I'm new to the process. What steps should I take to share those urls? Apologies if there's a standard process for getting data moved from use.yt/upload/ to yt-project/data that I'm unaware of.

When you upload data to use.yt/upload it returns the url that other people can use to download the data, e.g.:

xarth@shakuras /tmp $ echo "hello" > test.txt
xarth@shakuras /tmp $ curl -T test.txt http://use.yt/upload/
Stream body handler: received 6 bytes
http://use.yt/upload/392c289b
xarth@shakuras /tmp $ rm -rf test.txt && curl -JO http://use.yt/upload/392c289b
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     6  100     6    0     0    122      0 --:--:-- --:--:-- --:--:--   122
curl: Saved to filename 'test.txt'
xarth@shakuras /tmp $ cat test.txt
hello

See that http://use.yt/upload/392c289b ? That's what you share. Do you still have those urls?

@Xarthisius
Copy link
Member

I looked into database and identified them by size. FTR:

$ curl -JO http://use.yt/upload/ab645a18
curl: Saved to filename 'F37_80.tar.gz'
$ curl -JO http://use.yt/upload/f9c85197
curl: Saved to filename 'C15-3D-3deg.tar.gz'

@Xarthisius Xarthisius merged commit b8c7672 into yt-project:master Mar 31, 2022
@Ronan-Hix
Copy link
Contributor Author

Those are indeed the urls that curl passed out, I had just pulled them back up. Many thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants