We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Currently, a choice is closed even when an exception was thrown during its generation:
ai-dial-sdk/aidial_sdk/chat_completion/choice.py
Lines 55 to 62 in 8be932c
This leads to confusing streams of chunks like this one:
data: {"choices":[{"index":0,"finish_reason":null,"delta":{"role":"assistant"}}],"usage":null,"id":"7b8ef0df-26a6-4382-866a-98cf0a36bd1c","created":1732815811,"object":"chat.completion.chunk"} data: {"choices":[{"index":0,"finish_reason":"stop","delta":{}}],"usage":null,"id":"7b8ef0df-26a6-4382-866a-98cf0a36bd1c","created":1732815811,"object":"chat.completion.chunk"} data: {"error":{"message":"Something went wrong!","type":"internal_server_error"}} data: [DONE]
This creates a false impression that the generation of a choice was successful, but something went wrong later, after the choice was generated.
Note that the stages are closed with status=failed in cases of an exception:
status=failed
ai-dial-sdk/aidial_sdk/chat_completion/stage.py
Lines 48 to 60 in 8be932c
for which this no analog in the choice API.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Currently, a choice is closed even when an exception was thrown during its generation:
ai-dial-sdk/aidial_sdk/chat_completion/choice.py
Lines 55 to 62 in 8be932c
This leads to confusing streams of chunks like this one:
This creates a false impression that the generation of a choice was successful, but something went wrong later, after the choice was generated.
Note that the stages are closed with
status=failed
in cases of an exception:ai-dial-sdk/aidial_sdk/chat_completion/stage.py
Lines 48 to 60 in 8be932c
for which this no analog in the choice API.
The text was updated successfully, but these errors were encountered: