Listening to the Database's Own Journal
A more robust, modern approach is Change Data Capture (CDC). The core idea is elegant: instead of the application trying to write to two places, let the application write only to the legacy database — exactly as it always has, with zero code changes.
Every relational database maintains a transaction log (also called a Write-Ahead Log or WAL in PostgreSQL, the binlog in MySQL, or the redo log in Oracle). This log is an append-only record of every change: every INSERT, UPDATE, and DELETE. The database uses it internally for crash recovery and replication.
A CDC tool hooks into this transaction log and converts each change into a structured event. That event is published to a message broker (like Apache Kafka). Your new service subscribes to the relevant topic on the broker and updates its own database accordingly.
Here's the flow:
User Request → Legacy App → Legacy DB (writes normally)
↓
Transaction Log (WAL)
↓
CDC Tool (Debezium)
↓
Message Broker (Kafka)
↓
New Service → New DB
Debezium is the most widely used open-source CDC platform. You point it at the legacy database and tell it which tables to watch — say, the users and user_preferences tables — and it publishes every change to those tables as a message on a corresponding Kafka topic, with zero changes to the legacy application's code.
Each change event carries the row's state both before and after the change, along with metadata like the source table, timestamp, and the operation type (create, update, delete, or an initial read from the snapshot taken when the connector first starts). Having both the before and after state on every event is invaluable for debugging and auditing — you can see exactly what changed and when, not just that something changed.
Benefits of CDC over Dual Writes:
| Aspect | Dual Writes | CDC |
|---|---|---|
| Legacy code changes | Required | None |
| Consistency guarantee | Weak (can diverge silently) | Strong (log is source of truth) |
| Failure handling | Complex retry logic in app code | Messages queue in Kafka automatically |
| Latency of sync | Synchronous (adds latency to writes) | Asynchronous (typically < 1 second) |
| Audit trail | None built-in | Full ordered event log in Kafka |
| Operational complexity | In application code | In infrastructure (Debezium + Kafka) |
If asked about data migration strategies, mention CDC by name and reference Debezium. Then explain why it's superior to dual writes: "CDC is non-invasive — it reads the database's own transaction log, so you don't modify any legacy code. It's also more reliable because Kafka acts as a durable buffer; if the consumer is temporarily down, events queue up rather than being lost." This shows you understand both the mechanism and the trade-off reasoning, not just the tool name.
Key Point: CDC introduces eventual consistency — there is a small delay (typically milliseconds to low seconds) between a write to the legacy DB and the new service's database reflecting that change. For most use cases this is acceptable, but for scenarios requiring strict read-after-write consistency (e.g., a user updates their email and immediately views their profile via the new service), you may need to implement a "read-your-own-writes" pattern, where the new service checks the legacy DB directly for very recent changes, or uses a version token to detect stale reads. Also watch for replication slot lag in PostgreSQL: if the Debezium consumer falls behind, the WAL cannot be truncated, and the legacy database disk can fill up.