Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

429 Too Many Requests downloading thingiverse data #37

Open
eliphatfs opened this issue Apr 13, 2024 · 2 comments
Open

429 Too Many Requests downloading thingiverse data #37

eliphatfs opened this issue Apr 13, 2024 · 2 comments

Comments

@eliphatfs
Copy link

With only a single thread set, I'm still getting too many requests after downloading only a few (~100) objects.
It seems impossible to download 3M thingiverse data in sensible time.

@nallath
Copy link

nallath commented Jun 26, 2024

Sorry for that, but the huge amount of requests that scrapers like this are causing us have forced us to implement rather strict rate limiting. I was already wondering why I was seeing so many scrapers coming from universities as of a few months.

@feiqiu-cyber
Copy link

Sorry for that, but the huge amount of requests that scrapers like this are causing us have forced us to implement rather strict rate limiting. I was already wondering why I was seeing so many scrapers coming from universities as of a few months.

Hello,When I download for a while, I get 403 error,Is my IP banned permanently or for a period of time? How long is it banned?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants