bug: retry does not always have retry-after header

[dciborow]

gpt-review/src/gpt_review/_openai.py

Line 114 in 714d9ff

_retry_with_exponential_backoff(retry, error.headers["Retry-After"])

Sometimes the error will not contain the retry-after header.

openai.error.RateLimitError: The server is currently overloaded with other requests. Sorry about that! You can retry your request, or contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 if the error persists.

https://github.com/microsoft/gpt-review/actions/runs/4994531239/jobs/8945207747#step:7:191

[dciborow]

Request

vi issue.md
ISSUE=$(cat issue.md)
gpt ask --dir src/gpt_review/ "$ISSUE determine which file must be updated to fix this issue, and suggest a way to fix it"

Response

To fix this issue, you should update the _call_gpt() function in the _openai.py file. The function currently assumes the presence of a Retry-After header in the error object, but as the error message suggests, it might not always be present. You can add a fallback value for the retry_after variable in case the header is missing.

Here's a suggestion on how to handle this:

def _call_gpt(prompt, temperature, max_tokens, top_p, frequency_penalty, presence_penalty, retry=0):
    try:
        # ... (rest of the function)
    except RateLimitError as error:
        if retry < C.MAX_RETRIES:
            retry_after = error.headers.get("Retry-After", C.DEFAULT_RETRY_AFTER)
            _retry_with_exponential_backoff(retry, retry_after)

            return _call_gpt(prompt, temperature, max_tokens, top_p, frequency_penalty, presence_penalty, retry + 1)
        raise RateLimitError("Retry limit exceeded") from error

In this example, I've changed the line that retrieves the Retry-After header to use the get() method, which allows providing a default value (C.DEFAULT_RETRY_AFTER) if the header is not present. Make sure to define C.DEFAULT_RETRY_AFTER with a suitable default value in the configuration/constants module.