Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Does Azure Open AI Client handle concurrent requests for chat scenario #47298

Open
SamGuoMsft opened this issue Nov 22, 2024 · 1 comment
Open

Does Azure Open AI Client handle concurrent requests for chat scenario #47298

SamGuoMsft opened this issue Nov 22, 2024 · 1 comment
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team OpenAI question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@SamGuoMsft
Copy link

SamGuoMsft commented Nov 22, 2024 •
edited
Loading

Library name and version

Azure.AI.OpenAI 2.0.0

Query/Question

I keep a single AzureOpenAIClient, and create ChatClient for each request (underlying they shared the same pipeline based on looking at the code). When there are concurrent requests coming, I found sometimes the response for request A actually also returns to request B. Is this expected behavior? If yes, how I should handle this?

Environment

No response
@github-actions github-actions bot added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team OpenAI question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team. labels Nov 22, 2024
@github-actions GitHub Actions
Copy link

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @ralph-msft @trrwilson.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team OpenAI question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@SamGuoMsft