[Github] Fix Rate limit error #2711
Conversation
…hub-fix-rate-limit-error
There was a problem hiding this comment.
Sorry for the delay in reviewing.
I want to confirm, does this PR actually close the issue #2636 ? Because reading that issue, it sounds like a somewhat different problem.
| if self.get_rate_limit_encountered(exception.status_code, exception): | ||
| await self._put_to_sleep(resource_type="graphql") |
There was a problem hiding this comment.
Why do we no longer want to check rate limiting after a BadGraphQLRequest exception?
There was a problem hiding this comment.
This function raises QueryError instead of BadGraphQLRequest when rate limit is encountered.
connectors/sources/github.py
Outdated
| ) | ||
| return set() | ||
| return {scope.strip() for scope in scopes.split(",")} | ||
| except Exception as exception: |
There was a problem hiding this comment.
Is there not a specific exception to capture here? I think it's worthwhile to separate specific error exceptions and unexpected exceptions
connectors/sources/github.py
Outdated
|
|
||
| return list(invalid_repos) | ||
| return list(invalid_repos) | ||
| except Exception as exception: |
There was a problem hiding this comment.
Same as above; is there a specific exception we can try to catch first?
I have updated the description; added #2636 for |
|
Since it's a bugfix, adding a label for the most recent minor to ensure a backport. |
💔 Failed to create backport PR(s)
To backport manually run: |
|
@moxarth-elastic can you run the ☝️ above command, resolve the conflicts, and be sure to open the 8.15 backport? |
Backport PR Link - #2746 |
Closes #2572
Relates to #2636
This PR fixes the rate limiting issue in the Github connector while facing a 403 status code (Forbidden). The connector is able to fetch the documents after token has hit the rate limit.
Checklists
Pre-Review Checklist
config.yml.example)v7.13.2,v7.14.0,v8.0.0)Release Note
Fixing the rate limit issue while facing 403 - Forbidden error.