CDC Operations
This page covers the operational side of CDC: what to monitor, what restart means, and where retention can break an otherwise correct pipeline.
What to monitor
- Source lag -- is the source resume position continuing to advance?
- Destination apply rate -- can the destination keep up with the incoming mutation volume?
- Retention window -- will the source still retain the required log or stream history if the runner is down?
- Schema and key validation -- do business keys and expected columns still match the source?
Restart expectations
- If the runner stops after a committed change batch, restart resumes from the last committed ownership point.
- If failure happens before commit, the source may be reread, but final-state destinations remain idempotent because reconciliation happens on apply.
- A healthy restart still depends on retention: the source must still have the WAL, binlog, stream, or topic data needed to resume.
Retention checklist by source
| Source | Operational requirement |
|---|---|
| PostgreSQL | Logical replication slot must not fall behind so far that required WAL is removed |
| MySQL | Binlog retention must cover your worst-case outage window |
| MongoDB | Oplog window must cover your worst-case outage window |
| DynamoDB | Stream retention must be long enough for expected downtime and catch-up |
| Kafka | Topic retention and consumer-group offsets must outlive the outage window |
Backfill vs resume
- Resume -- use a normal restart when you want to continue from the last committed ownership point.
- Backfill -- reset or create a new project when you want to rebuild from scratch.
- Contract changes -- if business keys or reconciliation assumptions change, treat that as a migration rather than an ordinary restart.
Common warning signs
| Symptom | Likely cause |
|---|---|
| Lag keeps growing | Destination apply path is slower than source change volume |
| Resume fails after downtime | Source retention expired before the runner came back |
| Deletes appear wrong | Business keys or tombstone assumptions are incorrect |
| CDC starts but stops at startup validation | Source and destination capabilities do not support the requested contract |