You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, AI-related plugins only have a timeout parameter. After reviewing the code, I found that timeout is the total of connect, read, and send timeout durations, making it impossible to precisely control the timeout for requests to the LLM service. This needs to be improved.
Description
Currently, AI-related plugins only have a timeout parameter. After reviewing the code, I found that timeout is the total of connect, read, and send timeout durations, making it impossible to precisely control the timeout for requests to the LLM service. This needs to be improved.
Currently https://github.com/apache/apisix/blob/master/apisix/plugins/ai-drivers/openai-base.lua#L70
Expected:
The text was updated successfully, but these errors were encountered: