We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
需求:需要在抓取数据的请求超时之后,能自动切换到一个新的代理ip,因为在我们的场景下,代理ip是有时效的,存活大概就2~3分钟左右。看了很多文档和问题,推荐的做法是自定义一个proxyProvider来实现动态切换代理。 问题:我在proxyProvider里怎么才能能知道上次的请求已经超时了呢?
The text was updated successfully, but these errors were encountered:
returnProxy 的入参有 Page 和 Task,可以获得上次请求的情况: us.codecraft.webmagic.proxy.ProxyProvider.returnProxy(Proxy, Page, Task)
us.codecraft.webmagic.proxy.ProxyProvider.returnProxy(Proxy, Page, Task)
Sorry, something went wrong.
No branches or pull requests
需求:需要在抓取数据的请求超时之后,能自动切换到一个新的代理ip,因为在我们的场景下,代理ip是有时效的,存活大概就2~3分钟左右。看了很多文档和问题,推荐的做法是自定义一个proxyProvider来实现动态切换代理。
问题:我在proxyProvider里怎么才能能知道上次的请求已经超时了呢?
The text was updated successfully, but these errors were encountered: