You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, we deploy dagster on ECS and use PostgresRunStorage. We are trying to get tracing up for the daemon with dd-trace, but it logs a lot of error traces like :
/api.DagsterApi/Ping -> StatusCode.UNAVAILABLE: failed to connect to all addresses; last error: UNKNOWN: unix:/tmp/tmpa9xdjdq7: No such file or directory
Which end up saturating the process.
I think this originates in this part of the code :
defwait_for_grpc_server(server_process, client, subprocess_args, timeout=60):
start_time=time.time()
last_error=NonewhileTrue:
try:
client.ping("")
returnexceptDagsterUserCodeUnreachableError:
last_error=serializable_error_info_from_exc_info(sys.exc_info())
iftimeout>0and (time.time() -start_time>timeout):
raiseException(
f"Timed out waiting for gRPC server to start after {timeout}s with arguments:"f" \"{' '.join(subprocess_args)}\". Most recent connection error: {last_error}"
)
ifserver_process.poll() isnotNone:
raiseException(
f"gRPC server exited with return code {server_process.returncode} while starting up"f" with the command: \"{' '.join(subprocess_args)}\""
)
sleep(0.1)
Is this a dd-trace problem, or a dagster problem ? We have not done any particular configuration for the daemon's grpc server.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi, we deploy dagster on ECS and use PostgresRunStorage. We are trying to get tracing up for the daemon with dd-trace, but it logs a lot of error traces like :
/api.DagsterApi/Ping ->
StatusCode.UNAVAILABLE: failed to connect to all addresses; last error: UNKNOWN: unix:/tmp/tmpa9xdjdq7: No such file or directory
Which end up saturating the process.
I think this originates in this part of the code :
Is this a dd-trace problem, or a dagster problem ? We have not done any particular configuration for the daemon's grpc server.
Beta Was this translation helpful? Give feedback.
All reactions