Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rendering of ir-metadata #64

Open
mam10eks opened this issue Mar 25, 2025 · 3 comments
Open

Rendering of ir-metadata #64

mam10eks opened this issue Mar 25, 2025 · 3 comments

Comments

@mam10eks
Copy link
Member

Dear all (cc @hscells, @janheinrichmerker, @TheMrSheldon),

I prepared a first widget to render ir-metadata collected via the tirex-tracker.

My current example is to index a dataset into ChatNoir, and I uploaded the metadata here (i.e., this is the page where you could claim that an run upload is yours, this is the fastest way to play around with this at the moment): https://www.tira.io/claim-submission/32f56b61-875d-4446-8313-a94f565408de

I would like to use this as an running example to identify things that we want to improve.

Are there any things that come to your mind?

One thing that I would find very helpful would be to translate measurement units in the UI.

E.g., Currently we have things like:

    system: 49500 ms
    user: 477050 ms
    wallclock: 9897556 ms

I would maybe translate this to:

    system: 0.8 min (49500 ms)
    user: 7.9 min (477050 ms)
    wallclock: 164.9 min (9897556 ms)

Thanks in advance!

@janheinrichmerker
Copy link
Contributor

Super awesome!
I played around with the metadata UI for a short while. Here's some thoughts on it:

  • The plots could show the units directly both as absolute values (e.g., 100kb) and as percentage of the total available resource (e.g., 10%). I think this would be super useful to identify situations where CPU is throttled and where OOMs could likely occur. (Later TIRA could even give "hints" to researchers as to how they could improve their software.)
  • The remaining metadata could gradually be moved to Vuetify nested lists (https://dev.vuetifyjs.com/en/components/lists/#sub-group) to make the metadata more readable to less techy persons
  • Or maybe some mini dashboard similar to this (https://dev.vuetifyjs.com/en/components/sparklines/#dashboard-card) where researchers who want to reuse a component can directly see what hardware they'd need

Altogether there is just so much in the metadata to make use of, extremely nice IMO!

@janheinrichmerker
Copy link
Contributor

Ah, and one quick thought: The "dashboard" could roughly be structured along the PRIMAD dimensions, so that someone only interested in the method can easily skip to the "Method" section

@janheinrichmerker
Copy link
Contributor

Once we have metadata for more components, this will be totally awesome as then we can also render some "overview" dashboards in the sense of "X submissions of this shared task used a GPU", "this shared task costs XX$ to replicate" or similar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants