[BUG]: Unable to use asyncpg in new library version #11852
Comments
|
The error message in your log occurs in this part of the code: dd-trace-py/ddtrace/_trace/tracer.py Lines 849 to 854 in a604d93 It looks like the affected spans are
It's not clear to me how these two spans would interact with asyncpg
Instead, I wonder if there's an interaction with Is there a staging environment where this issue occurs where you can try this with 2.20.0 to see if any of the asyncio changes fix it: https://github.com/DataDog/dd-trace-py/releases/tag/v2.20.0 ? Regarding:
I'm worried that if 2.20.0 doesn't show any specific improvements, we'll have to get the reproduction to figure out how to find the bug. In case that happens, I do recommend a support ticket, referencing this Github issue so I can look at it from the other side. Even if you can't create a small reproducible example, a small code repro that showcases how the app is configured, such as python tracer settings would be useful. Other factors that I'm wondering about that may require details better shared over our support portal vs this public forum:
|
|
Thanks for the detailed response @wantsui . I tried version Here's a small reproducible example: import asyncio
import datetime as dt
import freezegun
from ddtrace import patch_all
from sqlalchemy import text
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
_HOST = "localhost"
_PORT = 5432
_USERNAME = ""
_PASSWORD = ""
_DB_NAME = ""
_DB_STRING = f"postgresql+asyncpg://{_USERNAME}:{_PASSWORD}@{_HOST}:{_PORT}/{_DB_NAME}"
async def _run():
patch_all(asyncpg=True)
engine = create_async_engine(_DB_STRING, isolation_level="AUTOCOMMIT")
async_session = async_sessionmaker(engine, expire_on_commit=False)
with freezegun.freeze_time(dt.datetime.now()):
async with async_session() as session:
result = await session.execute(text("select 1"))
assert result.all() == [(1,)]
print("Success!")
if __name__ == "__main__":
asyncio.run(_run())This code connects to a Postgres database and executes Do you get the same result when running this code? |
|
Hi @Apakottur It looks like this error starts happening at tracer version While we investigate further, could you change |
|
Thanks for explanation @quinna-h ! I tried So far this also seems to work well on our production environment, I'll update if we hit further issues. |
Tracer Version(s)
2.18.1
Python Version(s)
3.12.8
Pip Version(s)
pip 24.3.1
Bug Report
We are tracing our application which communicates with postgres via the asyncpg driver.
Everything works perfectly in ddtrace==2.17.3 which we are currently using, but upgrading to 2.18.1 results in our asyncpg driver constantly timing out, for example:
Reproduction Code
So far I was not able to create a small reproducible example, this happens after the process runs for a while, performing many different DB queries. Will continue trying.
Error Logs
We're getting the following debug log from the tracer when the issue occurs:
Libraries in Use
aiobotocore==2.16.0
aiohappyeyeballs==2.4.4
aiohttp==3.11.11
aioitertools==0.12.0
aiosignal==1.3.2
amplitude-analytics==1.1.4
annotated-types==0.7.0
anyio==4.7.0
asgi-lifespan==2.1.0
asyncpg==0.30.0
asyncpg-stubs==0.30.0
attrs==24.3.0
boto3==1.35.81
botocore==1.35.81
botocore-stubs==1.35.90
bytecode==0.16.0
CacheControl==0.14.1
cachetools==5.5.0
certifi==2024.12.14
cffi==1.17.1
charset-normalizer==3.4.1
click==8.1.8
coverage==7.6.10
cryptography==44.0.0
ddtrace==2.18.1
Deprecated==1.2.15
dnspython==2.7.0
email_validator==2.2.0
emoji==2.14.0
envier==0.6.1
execnet==2.1.1
firebase-admin==6.6.0
freezegun==1.5.1
frozenlist==1.5.0
GeoAlchemy2==0.16.0
geographiclib==2.0
geopandas==1.0.1
geopy==2.4.1
google-api-core==2.24.0
google-api-python-client==2.156.0
google-auth==2.37.0
google-auth-httplib2==0.2.0
google-cloud-core==2.4.1
google-cloud-firestore==2.19.0
google-cloud-storage==2.19.0
google-crc32c==1.6.0
google-resumable-media==2.7.2
googleapis-common-protos==1.66.0
greenlet==3.1.1
grpcio==1.68.1
grpcio-status==1.68.1
h11==0.14.0
h3==4.1.2
httpcore==1.0.7
httplib2==0.22.0
httptools==0.6.4
httpx==0.28.1
idna==3.10
importlib_metadata==8.5.0
iniconfig==2.0.0
Jinja2==3.1.5
jmespath==1.0.1
jsonschema==4.23.0
jsonschema-path==0.3.3
jsonschema-specifications==2023.12.1
lazy-object-proxy==1.10.0
loguru==0.7.3
markdown-it-py==3.0.0
MarkupSafe==3.0.2
mdurl==0.1.2
moto==5.0.24
msgpack==1.1.0
multidict==6.1.0
mypy==1.14.0
mypy-extensions==1.0.0
nodeenv==1.9.1
numpy==2.2.1
openapi-schema-validator==0.6.2
openapi-spec-validator==0.7.1
opentelemetry-api==1.29.0
packaging==24.2
pandas==2.2.3
pandas-stubs==2.2.3.241126
pathable==0.4.3
pgpasslib==1.1.0
phonenumbers==8.13.52
pluggy==1.5.0
propcache==0.2.1
proto-plus==1.25.0
protobuf==5.29.2
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycparser==2.22
pydantic==2.10.4
pydantic-settings==2.7.0
pydantic_core==2.27.2
Pygments==2.18.0
pyinstrument==5.0.0
PyJWT==2.10.1
pyogrio==0.10.0
pyparsing==3.2.1
pyproj==3.7.0
pyright==1.1.391
pytest==8.3.4
pytest-asyncio==0.25.0
pytest-cov==6.0.0
pytest-httpx==0.35.0
pytest-mock==3.14.0
pytest-sugar==1.0.0
pytest-xdist==3.6.1
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
pytz==2024.2
PyYAML==6.0.2
referencing==0.35.1
requests==2.32.3
responses==0.25.3
rfc3339-validator==0.1.4
rich==13.9.4
rpds-py==0.22.3
rsa==4.9
s3transfer==0.10.4
shapely==2.0.6
shellingham==1.5.4
shpyx==0.0.32
six==1.17.0
sniffio==1.3.1
SQLAlchemy==2.0.35
starlette==0.43.0
tenacity==9.0.0
termcolor==2.5.0
timezonefinder==6.5.7
typeguard==4.4.1
typer==0.15.1
types-aiobotocore==2.16.0
types-aiobotocore-personalize-runtime==2.16.1
types-aiobotocore-rekognition==2.16.1
types-aiobotocore-s3==2.16.1.post1
types-aiobotocore-sqs==2.16.1
types-aiobotocore-ssm==2.16.1
types-aiobotocore-stepfunctions==2.16.1
types-awscrt==0.23.6
types-python-dateutil==2.9.0.20241206
types-pytz==2024.2.0.20241221
types-shapely==2.0.0.20241221
typing_extensions==4.12.2
tzdata==2024.2
uritemplate==4.1.1
urllib3==2.3.0
uvicorn==0.34.0
uvloop==0.21.0
watchfiles==1.0.3
websockets==14.1
Werkzeug==3.1.3
wrapt==1.17.0
xmltodict==0.14.2
yarl==1.18.3
zipp==3.21.0
Operating System
Linux 3a14a4cc78d4 6.10.14-linuxkit #1 SMP PREEMPT_DYNAMIC Fri Nov 29 17:24:06 UTC 2024 x86_64 GNU/Linux
The text was updated successfully, but these errors were encountered: