Replies: 1 comment
-
I've opened #24834 to support sidecars for Dagster pods. There are some workarounds required for Kubernetes versions before 1.29, but this PR should allow folks to deploy Dagster on GKE with Cloud SQL Auth Proxy for versions both before and after 1.29. K8s 1.29+ K8s 1.29+ Example Config
K8s before 1.29 K8s 1.28 Example Config
|
Beta Was this translation helpful? Give feedback.
-
I have Dagster deployed on Google Kubernetes Engine using the Dagster provided Helm chart. In my deployment, I've disabled PostgreSQL in favor of using Google Cloud SQL. I felt that using a managed database service for my database would be wise given Google handles things like automated backups, point-in-time recovery, automated patching and updates, and the ability to easily adjust compute and memory.
Today I have my values.yaml configured to use the private IP of my Cloud SQL instance (see here). This works and it keeps the traffic internal, but it does not use SSL. Not great.
Google recommends (see here) the use of Cloud SQL Auth proxy to connect to Cloud SQL. This allows for an encrypted connection between the GKE cluster and Cloud SQL. In this architecture, the Cloud SQL Auth proxy is added to the pods using the sidecar container pattern. Dagster's daemon and Dagit would then connect to the PostgreSQL instance via localhost.
Not a Dagster example, but you will see here an example of when I had to do this with a different application.
https://github.com/xmarcosx/dagster-via-kubernetes
It would be so fantastic if Dagster added support for this sidecar pattern to allow for an easy way to deploy Dagster in GKE while keeping the database in Cloud SQL.
Beta Was this translation helpful? Give feedback.
All reactions