Skip to content

ERR Replication error occurred error="receive message failed: unexpected EOF" #89

@koltyakov

Description

@koltyakov

I'm checking if pg_flo suites my pipeline. Testing with docker-compose from the sample and configuration based on https://github.com/pgflo/pg_flo/blob/main/internal/pg-flo.yaml (with source and target connections and creds).

  • pg_flo: v0.0.13 (from releases)
  • OS: macOS Sequoia (apple silicon)
  • Replicator: ./pg_flo replicator --config ./pg-flo.yml
  • Worker: ./pg_flo worker postgres --config ./pg-flo.yml
  • Super user account
  • Postgres images versions: postgres:17

DDLs are synced manually, then I add/update/delete rows and target DB receives CDC updates. Checked with few updates, then 80K, that works.

However, when I run a bulk change (e.g. insert a million of dummy records):

/*
CREATE TABLE users (
  id SERIAL PRIMARY KEY,
  name TEXT
);

CREATE PUBLICATION pg_flo_group FOR TABLE users;
*/

INSERT INTO users (name)
SELECT gen_random_uuid()::TEXT
FROM generate_series(1, 10000000);

Data is synced, I see all rows appear in target, yet replicator fails with "receive message failed: unexpected EOF" and exits the process:

16:24:44.904 DBG Sent standby status update component=replicator lsn=0/6542E2D0
16:24:44.905 INF Relation message received component=replicator id=16390 table=users
16:24:44.983 ERR Error processing next message error="receive message failed: unexpected EOF" component=replicator
16:24:44.983 ERR Replication error occurred error="receive message failed: unexpected EOF"
16:24:44.983 INF Initiating graceful shutdown component=replicator
16:24:44.983 DBG Sent standby status update component=replicator lsn=0/6542E2D0
16:24:44.984 INF Closing database connections component=replicator
16:24:44.984 INF Base replicator shutdown completed component=replicator

On replicator restart, it sends messages again from the beginning and then fails again with the same error.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions