Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception when using AsyncQdrantClient #806

Open
gushinetsm opened this issue Oct 9, 2024 · 0 comments
Open

Exception when using AsyncQdrantClient #806

gushinetsm opened this issue Oct 9, 2024 · 0 comments

Comments

@gushinetsm
Copy link

I`m using AsyncQdrantClient to insert embeddings into QDrant and I invoke this piece of python code from C++ multithreaded code (using pybind11 to communicate between C++ and Python code). Before invoking Python code from C++ I make sure to lock GIL and after execution returns back GIL is released. So the issue is following if I use QdrantClient it works fine no matter how many threads I run in C++ code (tested 1-20 threads). When I start using AsyncQdrantClient with only one thread in C++ it also works fine. But as soon as I increase number of threads in C++ code my python code starts failing randomly with the following exception. The more threads I use the often I get these exceptions:

Exception has occurred: ResponseHandlingException       (note: full exception trace is shown but execution is paused at: writeTextEnd)
list.remove(x): x not in list
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\qdrant_client\http\api_client.py", line 173, in send_inner
    response = await self._async_client.send(request)
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpx\_client.py", line 1660, in send
    raise exc
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpx\_client.py", line 1654, in send
    await response.aread()
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpx\_models.py", line 915, in aread
    self._content = b"".join([part async for part in self.aiter_bytes()])
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpx\_models.py", line 915, in <listcomp>
    self._content = b"".join([part async for part in self.aiter_bytes()])
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpx\_models.py", line 933, in aiter_bytes
    async for raw_bytes in self.aiter_raw():
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpx\_models.py", line 999, in aiter_raw
    await self.aclose()
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpx\_models.py", line 1012, in aclose
    await self.stream.aclose()
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpx\_client.py", line 153, in aclose
    await self._stream.aclose()
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpx\_transports\default.py", line 257, in aclose
    await self._httpcore_stream.aclose()
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpcore\_async\connection_pool.py", line 378, in aclose
    closing = self._pool._assign_requests_to_connections()
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\httpcore\_async\connection_pool.py", line 258, in _assign_requests_to_connections
    self._connections.remove(connection)
ValueError: list.remove(x): x not in list

During handling of the above exception, another exception occurred:

  File "C:\Aparavi\aparavi-connectors\lib\site-packages\qdrant_client\http\api_client.py", line 175, in send_inner
    raise ResponseHandlingException(e)
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\qdrant_client\http\api_client.py", line 195, in __call__
    return await call_next(request)
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\qdrant_client\http\api_client.py", line 163, in send
    response = await self.middleware(request, self.send_inner)
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\qdrant_client\http\api_client.py", line 146, in request
    return await self.send(request, type_)
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\qdrant_client\http\api\points_api.py", line 760, in batch_update
    return await self._build_for_batch_update(
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\qdrant_client\async_qdrant_remote.py", line 1646, in batch_update_points
    await self.openapi_client.points_api.batch_update(
  File "C:\Aparavi\aparavi-connectors\lib\site-packages\qdrant_client\async_qdrant_client.py", line 1402, in batch_update_points
    return await self._client.batch_update_points(
  File "C:\Aparavi\aparavi-connectors\ai\providers\store\qdrant.py", line 500, in flush
    await self._client.batch_update_points(
  File "C:\Aparavi\aparavi-connectors\ai\providers\store\qdrant.py", line 545, in aaddChunks
    await flush()
  File "C:\Aparavi\aparavi-connectors\ai\common\store.py", line 259, in aadd
    await self.aaddChunks(chunks)
  File "C:\Aparavi\aparavi-connectors\lib\asyncio\base_events.py", line 646, in run_until_complete
    return future.result()
  File "C:\Aparavi\aparavi-connectors\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Aparavi\aparavi-connectors\ai\common\store.py", line 273, in add
    return asyncio.run(self.aadd(
  File "C:\Aparavi\aparavi-connectors\connectors\vectorize\IInstance.py", line 180, in writeTextEnd (Current frame)
    self.IGlobal.store.add(
qdrant_client.http.exceptions.ResponseHandlingException: list.remove(x): x not in list

Python version: Python 3.10.11
Qdrant version: 1.7.4, 1.11.4
Qdrant client version: 1.7.3, 1.11.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant