Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

making parallel most part of the featurwize #128

Closed
reza1615 opened this issue Jan 29, 2025 · 3 comments
Closed

making parallel most part of the featurwize #128

reza1615 opened this issue Jan 29, 2025 · 3 comments

Comments

@reza1615
Copy link
Contributor

My dataset has 4500 features and for selecting features the initial part before xgboost takes around 1 hour to run. I realized most part of the featurwize is not parallel is it possible to make them parallel?
@AutoViML
Copy link
Owner

Wherever I could use n_jobs=-1 I have used it. Other than that, I have not used multithreading which is now available in XGBoost. This could be something to think about.
Ram

@reza1615
Copy link
Contributor Author

reza1615 commented Jan 29, 2025 •
edited
Loading

we should inspect each part of the pipeline regarding to time to see how we can reduce the timing by vectorize or parallelize with Joblib or similar libraries

@AutoViML
Copy link
Owner

yes that is doable but I don't have the time. If you or anyone is interested I can help out.
Ram

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@reza1615 @AutoViML