Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SparkClient单实例多并发时产生混杂输出 #26

Open
artorias-gh opened this issue May 22, 2024 · 3 comments
Open

SparkClient单实例多并发时产生混杂输出 #26

artorias-gh opened this issue May 22, 2024 · 3 comments

Comments

@artorias-gh
Copy link

_SparkLLMClient在实例级别维护了一个接收大模型socket信息片段的queue。在多线程模式下共用同一_SparkLLMClient实例进行大模型调用时,并发触发on_message回调导致大模型回复片段按不确定时序被丢到同一个queue里,最终产生混杂的输出。

此外,目前似乎只实现了基于线程的异步,希望能加入基于事件的异步支持。

@whybeyoung
Copy link
Contributor

确实是这样,已知问题里说明了这点,近期我们会评估下你说的基于事件的异步支持,当然我们也欢迎有能力的同学pr。非常欢迎~

@BrightXiaoHan
Copy link

请问什么时候能解决,我单线程调用也会有这个问题。

@ATP-BME
Copy link

ATP-BME commented Jul 31, 2024

遇到相同问题,通常出现在失败一次之后,再次调用会产生混杂输出

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants