Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a user, I want to sacrifice resources for faster processing #774

Open
al-niessner opened this issue Nov 30, 2023 · 0 comments
Open

As a user, I want to sacrifice resources for faster processing #774

al-niessner opened this issue Nov 30, 2023 · 0 comments

Comments

@al-niessner
Copy link
Contributor

Checked for duplicates

Yes - I've already checked

πŸ§‘β€πŸ”¬ User Persona(s)

Define the number of processors/threads to use in processing. When validating a large bundle like in NASA-PDS/harvest#143 it takes 12325 seconds or 3:45 hours while only using about 40% of CPU and memory each. Halving the processing time would be wonderful.

πŸ’ͺ Motivation

Efficiency of human resources vs computer resources.

πŸ“– Additional Details

Is it even possible to run validate using multiple threads/processors? It may be its current architecture is limited to single thread/processor. Evaluate its ability or amount of change required to move to multi processing then revisit this issue.

Acceptance Criteria

Given
When I perform
Then I expect

βš™οΈ Engineering Details

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: ToDo
Development

No branches or pull requests

2 participants