doc: tmp use static envoy config in docker

hmoazzem · hmoazzem · commit 4b6061ae0fbc · 2025-03-14T21:24:19.000+01:00
Signed-off-by: hmoazzem &lt;moazzem@edgeflare.io&gt;
diff --git a/README.md b/README.md
@@ -4,44 +4,36 @@
 [![CodeQL](https://github.com/edgeflare/edge/actions/workflows/codeql.yml/badge.svg)](https://github.com/edgeflare/edge/actions/workflows/codeql.yml)
 [![Release](https://github.com/edgeflare/edge/actions/workflows/release.yml/badge.svg)](https://github.com/edgeflare/edge/actions/workflows/release.yml)
 
-Edge configures and manages: 
+edge configures and manages:
+
+* [PostgreSQL](https://www.postgresql.org/): The world's most advanced open source database
 * [ZITADEL](https://github.com/zitadel/zitadel) - Centralized identity provider (OIDC)
 * [SeaweedFS](https://github.com/seaweedfs/seaweedfs) - S3-compatible object storage
 * [edgeflare/pgo](https://github.com/edgeflare/pgo) - PostgREST-compatible API and Debezium-compatible CDC
 * [NATS](https://nats.io) - Message streaming platform
 * [envoyproxy](https://github.com/envoyproxy/envoy) - Cloud-native high-performance edge/middle/service proxy
-## How it works
 
-Edge launches and configures these components to work together as a unified backend with PostgreSQL - similar to Supabase or Pocketbase. And with scaling capabilities.
+for a unified backend - similar to Firebase, Supabase, Pocketbase etc. And with scaling capabilities.
 
 ## Deployment options
 
-Edge can run as:
-- A single binary (embeds official component binaries)
-- [Docker compose](./docker-compose.yaml)
-- Kubernetes resources (follow this README)
-- Via a Kubernetes CRD named [Project](./example/project.yaml)
-
-This project is in the ideation stage. Edge configures/manages the four underlying tools to create a cohesive system.
+- A single binary (embeds official component binaries): WIP
+- [Docker compose](./docker-compose.yaml) or Kubernetes resources: follow this README
+- Via a Kubernetes CRD: [Project](./example/project.yaml)
 
-Interested in experimenting or contributing? See [CONTRIBUTING.md](./CONTRIBUTING.md).
+edge is in very early stage. Interested in experimenting or contributing? See [CONTRIBUTING.md](./CONTRIBUTING.md).
 
 ```sh
 git clone git@github.com:edgeflare/edge.git && cd edge
 ```
 
-This uses iam.example.local and api.example.local domains. Ensure they point to the Gateway IP (envoyproxy) eg by adding an entry to `/etc/hosts` like
-
-```sh
-127.0.0.1 api.example.local iam.example.local
-```
-
 ### [docker-compose.yaml](./docker-compose.yaml)
 
 Adjust the docker-compose.yaml
 
 - Free up port 80 from envoy for zitadel's initial configuration via management API which requires end-to-end HTTP/2 support.
-envoyproxy config in docker doesn't support (our xds-server incomplete) HTTP/2 yet; on [k8s](https://raw.githubusercontent.com/edgeflare/pgo/refs/heads/main/k8s.yaml) everything works fine.
+envoyproxy config in docker doesn't support (we gotta finish xds-server) HTTP/2 yet.
+On [k8s](https://raw.githubusercontent.com/edgeflare/pgo/refs/heads/main/k8s.yaml) everything works fine.
 
 ```yaml
   envoy:
@@ -63,41 +55,28 @@ docker compose up -d
 
 #### Use the centralized IdP for authorization in Postgres via `pgo rest` (PostgREST API)
 
-Any OIDC compliant Identity Provider (eg ZITADEL, Keycloak, Auth0) can be used.
+Configure ZITADEL. Adjust the domains in env vars, and in `internal/util/envoy/config.yaml`
 
 ```sh
-export ZITADEL_ISSUER=http://iam.example.local
-export ZITADEL_API=iam.example.local:80
+export ZITADEL_ISSUER=http://iam.192-168-0-121.sslip.io
+export ZITADEL_API=iam.192-168-0-121.sslip.io:80
 export ZITADEL_KEY_PATH=__zitadel-machinekey/zitadel-admin-sa.json
-export ZITADEL_JWK_URL=http://iam.example.local/oauth/v2/keys
+export ZITADEL_JWK_URL=http://iam.192-168-0-121.sslip.io/oauth/v2/keys
 ```
 
-Configure components eg create OIDC clients in ZITADEL etc
-
 ```sh
 go run ./internal/util/configure/...
 ```
 
-Once done, revert the ports (use 80 for envoy), and `docker compose restart`
-
-#### pgo rest
-
-Visit http://iam.example.local, login and regenerate client-secret for oauth2-proxy client in edge project. Then adjust `internal/util/pgo/config.yaml`
+The above go code creates, among others, an OIDC client which pgo uses for authN/authZ. Any OIDC compliant Identity Provider (eg , Keycloak, Auth0) can be used; pgo just needs the client credentials.
 
-> `pgo rest` container fails because of proxy issues. It can be run locally
+Once ZITADEL is configured, revert the ports (use 80 for envoy), and `docker compose down && docker compose up -d`
 
-```sh
-go install github.com/edgeflare/pgo@latest # or download from release page
-```
-##### PostgREST-compatible REST API
-
-```sh
-pgo rest --config internal/util/pgo/config.yaml --rest.pg_conn_string "host=localhost port=5432 user=pgo password=pgopw dbname=main sslmode=prefer"
-```
+Visit ZITADEL UI (eg at http://iam.192-168-0-121.sslip.io), login (see docker-compose.yaml) and regenerate client-secret for oauth2-proxy client in edge project. Then update `internal/util/pgo/config.yaml` with the values. Again, `docker compose down && docker compose up -d`
 
-###### realtime/replication eg sync users from auth-db to app-db
+#### `pgo rest`: PostgREST-compatible REST API
 
-Create table in sink-db. See pgo repo for more examples
+Create a table in app-db for REST and pipeline demo. See pgo repo for more examples
 
 ```sh
 PGUSER=postgres PGPASSWORD=postgrespw PGHOST=localhost PGDATABASE=main PGPORT=5432 psql
@@ -109,14 +88,22 @@ CREATE SCHEMA IF NOT EXISTS iam;
 CREATE TABLE IF NOT EXISTS iam.users (
   id TEXT DEFAULT gen_random_uuid()::TEXT PRIMARY KEY
 );
+
+-- wide-open for demo. use GRANT and RLS for granular ACL
+GRANT USAGE ON SCHEMA iam to anon;
+GRANT ALL ON iam.users to anon;
 ```
 
-Start pipeline
+`docker restart edge_pgo-rest_1` to reload schema cache if it bugs. Now we can for example
 
 ```sh
-pgo pipeline --config internal/util/pgo/config.yaml
+curl http://api.127-0-0-1.sslip.io/iam/users
 ```
 
+##### `pgo pipeline`: Debezium-compatible CDC for realtime-event/replication etc
+
+The demo pgo-pipeline container syncs users from auth-db (in projections.users14 table) to app-db (in iam.users)
+
 ### Kubernetes
 If you already have a live k8s cluster, great just copy-paste-enter.
 For development and lightweight prod, [k3s](https://github.com/k3s-io/k3s) seems a great option.
@@ -143,7 +130,6 @@ export ZITADEL_ADMIN_PW=$(kubectl get secrets example-zitadel-firstinstance -o j
 
 Configure zitadel like in docker-compose. Then apply something like `https://raw.githubusercontent.com/edgeflare/pgo/refs/heads/main/k8s.yaml`
 
-
 ## Cleanup
 
 ```sh
diff --git a/docker-compose.yaml b/docker-compose.yaml
@@ -34,7 +34,7 @@ services:
       ZITADEL_DATABASE_POSTGRES_ADMIN_USERNAME: postgres
       ZITADEL_DATABASE_POSTGRES_ADMIN_PASSWORD: postgrespw
       ZITADEL_DATABASE_POSTGRES_ADMIN_SSL_MODE: disable
-      ZITADEL_EXTERNALDOMAIN: iam.example.local
+      ZITADEL_EXTERNALDOMAIN: iam.192-168-0-121.sslip.io   # resolves to 192.168.0.121 (gateway/envoy IP). use LAN or accesible IP/hostname
       ZITADEL_EXTERNALPORT: 80
       ZITADEL_PORT: 8080
       ZITADEL_EXTERNALSECURE: false
@@ -73,6 +73,54 @@ services:
       timeout: 5s
       retries: 5
 
+  envoy-controlplane:
+    build:
+      context: "."
+      dockerfile: "./internal/util/envoy/Containerfile"
+    entrypoint: /envoy-controlplane
+    ports:
+    - 18000:18000 # xds-server
+    environment:
+    - "DEBUG=true"
+
+  envoy:
+    image: docker.io/envoyproxy/envoy:contrib-v1.33-latest
+    # command: [envoy, --config-path, /etc/bootstrap.yaml, --base-id, 1]   # when used with envoy-controlplane for dynamic config
+    command: [envoy, --config-path, /etc/config.yaml, --base-id, 1]
+    privileged: true
+    ports:
+    - 9901:9901      # admin
+    - 80:10080       # http-proxy
+    volumes:
+    # - $PWD/internal/util/envoy/bootstrap.yaml:/etc/bootstrap.yaml:rw,Z   # when used with envoy-controlplane
+    - $PWD/internal/util/envoy/config.yaml:/etc/config.yaml:rw,Z           # hard-coded config
+    depends_on:
+    - envoy-controlplane
+
+  pgo-rest:
+    image: ghcr.io/edgeflare/pgo
+    command: [rest, --config, /rest/config.yaml]
+    ports:
+    - 8080:8080
+    volumes:
+    - $PWD/internal/util/pgo/config.yaml:/rest/config.yaml:rw,Z
+    depends_on:
+      app-db:
+        condition: service_healthy
+      # init-app-db:
+      #   condition: service_completed_successfully # errors
+
+  pgo-pipeline:
+    image: ghcr.io/edgeflare/pgo
+    command: [pipeline, --config, /pipeline/config.yaml]
+    volumes:
+    - $PWD/internal/util/pgo/config.yaml:/pipeline/config.yaml:rw,Z
+    depends_on:
+      auth-db:
+        condition: service_healthy
+      app-db:
+        condition: service_healthy
+
   init-app-db:
     image: docker.io/bitnami/postgresql:17
     environment:
@@ -93,71 +141,37 @@ services:
         sleep 2
       done
       echo "PostgreSQL is up - executing SQL"
-
       psql -c "
-      DO \$\$
-      BEGIN
+        DO \$\$
+        BEGIN
           IF NOT EXISTS (
-              SELECT 1
-              FROM pg_roles
-              WHERE rolname = 'pgo'
+            SELECT 1
+            FROM pg_roles
+            WHERE rolname = 'pgo'
           ) THEN
-              CREATE ROLE pgo WITH LOGIN PASSWORD 'pgopw';
+            CREATE ROLE pgo WITH LOGIN PASSWORD 'pgopw';
           END IF;
           IF NOT EXISTS (
-              SELECT 1
-              FROM pg_roles
-              WHERE rolname = 'authn'
+            SELECT 1
+            FROM pg_roles
+            WHERE rolname = 'authn'
           ) THEN
-              CREATE ROLE authn NOLOGIN;
+            CREATE ROLE authn NOLOGIN;
           END IF;
           IF NOT EXISTS (
-              SELECT 1
-              FROM pg_roles
-              WHERE rolname = 'anon'
+            SELECT 1
+            FROM pg_roles
+            WHERE rolname = 'anon'
           ) THEN
-              CREATE ROLE anon NOLOGIN;
+            CREATE ROLE anon NOLOGIN;
           END IF;
           GRANT anon TO authn;
-          GRANT authn,anon to pgo;
-      END
-      \$\$;
+          GRANT authn to pgo;
+        END
+        \$\$;
       "
     restart: on-failure
 
-  envoy-controlplane:
-    build:
-      context: "."
-      dockerfile: "./internal/util/envoy/Containerfile"
-    entrypoint: /envoy-controlplane
-    ports:
-    - 18000:18000 # xds-server
-    environment:
-    - "DEBUG=true"
-
-  envoy:
-    image: docker.io/envoyproxy/envoy:contrib-v1.33-latest
-    command: [envoy, --config-path, /etc/bootstrap.yaml, --base-id, 1]
-    privileged: true
-    ports:
-    - 9901:9901      # admin
-    - 80:10080       # http-proxy
-    volumes:
-    - $PWD/internal/util/envoy/bootstrap.yaml:/etc/bootstrap.yaml:rw,Z
-    depends_on:
-    - envoy-controlplane
-
-  pgo-rest:
-    image: ghcr.io/edgeflare/pgo
-    command: [rest, --config, /rest/config.yaml]
-    ports:
-    - 8080:8080
-    volumes:
-    - $PWD/internal/util/pgo/config.yaml:/rest/config.yaml:rw,Z
-    depends_on:
-      app-db:
-        condition: service_healthy
-
 volumes:
   app-db:
   auth-db:
diff --git a/internal/util/envoy/config.yaml b/internal/util/envoy/config.yaml
@@ -0,0 +1,106 @@
+admin:
+  address:
+    socket_address:
+      address: 0.0.0.0
+      port_value: 9901
+
+static_resources:
+  listeners:
+  - name: listener_0
+    address:
+      socket_address:
+        address: 0.0.0.0
+        port_value: 10080
+    filter_chains:
+    - filters:
+      - name: envoy.filters.network.http_connection_manager
+        typed_config:
+          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
+          codec_type: auto
+          stat_prefix: ingress_http
+          route_config:
+            name: multi_service_route
+            virtual_hosts:
+            - name: iam_service
+              domains: ["iam.127-0-0-1.sslip.io", "iam.192-168-0-121.sslip.io", "iam.example.local", "zitadel.example.local"]
+              routes:
+              - match: { prefix: "/" }
+                route:
+                  cluster: iam_zitadel_service
+                  timeout: 0s
+                  max_stream_duration:
+                    grpc_timeout_header_max: 0s
+              cors:
+                allow_origin_string_match:
+                - prefix: "*"
+                allow_methods: GET, PUT, DELETE, POST, OPTIONS
+                allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
+                max_age: "1728000"
+                expose_headers: custom-header-1,grpc-status,grpc-message
+            - name: pgo_service
+              domains: ["api.127-0-0-1.sslip.io", "api.example.local", "pgo.example.local"]
+              routes:
+              - match: { prefix: "/" }
+                route:
+                  cluster: pgo_rest_service
+                  timeout: 0s
+                  max_stream_duration:
+                    grpc_timeout_header_max: 0s
+              cors:
+                allow_origin_string_match:
+                - prefix: "*"
+                allow_methods: GET, PUT, DELETE, POST, OPTIONS
+                allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
+                max_age: "1728000"
+                expose_headers: custom-header-1,grpc-status,grpc-message
+          http_filters:
+          - name: envoy.filters.http.grpc_web
+            typed_config:
+              "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_web.v3.GrpcWeb
+          - name: envoy.filters.http.cors
+            typed_config:
+              "@type": type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors
+          - name: envoy.filters.http.router
+            typed_config:
+              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
+
+  clusters:
+  - name: iam_zitadel_service
+    connect_timeout: 0.25s
+    type: LOGICAL_DNS
+    typed_extension_protocol_options:
+      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
+        "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
+        explicit_http_config:
+          # http_protocol_options: {}
+          http2_protocol_options: {}
+    lb_policy: round_robin
+    load_assignment:
+      cluster_name: iam_zitadel_cluster
+      endpoints:
+      - lb_endpoints:
+        - endpoint:
+            address:
+              socket_address:
+                address: iam-zitadel
+                port_value: 8080
+
+  - name: pgo_rest_service
+    connect_timeout: 0.25s
+    type: LOGICAL_DNS
+    typed_extension_protocol_options:
+      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
+        "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
+        explicit_http_config:
+          http_protocol_options: {}
+          # http2_protocol_options: {}
+    lb_policy: round_robin
+    load_assignment:
+      cluster_name: pgo_rest_cluster
+      endpoints:
+      - lb_endpoints:
+        - endpoint:
+            address:
+              socket_address:
+                address: pgo-rest
+                port_value: 8080
diff --git a/internal/util/pgo/config.yaml b/internal/util/pgo/config.yaml
