For production deployments, is it generally okay to have more than one dagster daemon instance? #17542
-
Quick question: For production deployments, is it generally okay to have more than one dagster daemon instance? From what I understand, they're used to kickoff various background tasks like sensors and auto-materialization of assets. What safeguards would be in place to handle race conditions between multiple dagster daemon instances such as two of them picking up the same task? The question was originally asked in Dagster Slack. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
No. "A deployment can have multiple simultaneous instances of dagster-webserver, but should include only a single dagster-daemon process." From Running the Dagster daemon documentation. Having multiple instances of the |
Beta Was this translation helpful? Give feedback.
No. "A deployment can have multiple simultaneous instances of dagster-webserver, but should include only a single dagster-daemon process." From Running the Dagster daemon documentation.
Having multiple instances of the
dagster-daemon
could lead to race conditions and other issues.