DBAtlas · Database operations atlas

Set up continuous replication / CDC (zero-downtime cutover)

Oracle

# CDC-based replication with Oracle GoldenGate (outline)
# 1) Enable supplemental logging on the source:
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
# 2) Extract (capture changes from redo) and Replicat (apply to the target) processes:
#   GGSCI> ADD EXTRACT ext1, INTEGRATED TRANLOG, BEGIN NOW
#   GGSCI> ADD REPLICAT rep1, INTEGRATED
# Alternative: initial load with Data Pump + incremental capture with GoldenGate

In Oracle, the standard tool for zero-downtime/heterogeneous CDC is Oracle GoldenGate: the Extract process captures changes from the redo logs and Replicat applies them to the target; supplemental logging must be enabled. Typical migration: the initial load is taken with Data Pump, and from then on GoldenGate streams changes to the target, enabling a cutover with near-zero downtime.

PostgreSQL

-- Source (publisher): create a publication (wal_level must be logical)
CREATE PUBLICATION migration_pub FOR ALL TABLES;

-- Target (subscriber): create a subscription (initial copy + continuous stream)
CREATE SUBSCRIPTION migration_sub
  CONNECTION 'host=source dbname=shop user=repl password=***'
  PUBLICATION migration_pub;

PostgreSQL provides CDC with built-in logical replication: CREATE PUBLICATION on the source, CREATE SUBSCRIPTION on the target. When the subscription starts it copies the existing data (initial copy), then continuously streams changes from WAL; wal_level=logical is required. For large/complex scenarios, pglogical or Debezium is preferred.

MySQL

-- Source: binlog must be enabled and in ROW format (log_bin, binlog_format=ROW)
-- Grant a replication user: GRANT REPLICATION SLAVE ON *.* TO 'repl'@'%';

-- Define the source on the target (start from the binlog position of the initial load):
CHANGE REPLICATION SOURCE TO
  SOURCE_HOST='source', SOURCE_USER='repl', SOURCE_PASSWORD='***',
  SOURCE_LOG_FILE='binlog.000001', SOURCE_LOG_POS=1547;
START REPLICA;

In MySQL, a zero-downtime cutover is done with binlog-based replication. The initial load is taken with mysqldump --source-data and the binlog position/GTID is recorded; on the target, replication is started from that position with CHANGE REPLICATION SOURCE TO (8.0.23+ syntax; formerly CHANGE MASTER TO). START REPLICA applies changes continuously. A GTID-based setup is safer.

SQL Server

-- Option A: Transactional Replication (publisher->distributor->subscriber) — SSMS wizard

-- Option B: Change Data Capture (track changes as tables)
EXEC sys.sp_cdc_enable_db;              -- enable CDC at database level
EXEC sys.sp_cdc_enable_table
  @source_schema = N'dbo', @source_name = N'customer',
  @role_name = NULL;                    -- enable CDC at table level
-- Changes are read from the cdc.dbo_customer_CT table and moved to the target.

SQL Server has two main paths for continuous replication: if you want a direct stream to the target, set up Transactional Replication (publisher/subscriber); if you will move changes with your own ETL, enable Change Data Capture (sp_cdc_enable_db + sp_cdc_enable_table) and read from the _CT tables. For heterogeneous targets, CDC is combined with AWS DMS/Debezium.

MongoDB

// Watch changes live with Change Streams (requires a replica set/sharded cluster)
const cs = db.customer.watch([], { fullDocument: "updateLookup" });
while (cs.hasNext()) { printjson(cs.next()); }  // apply to the target

// Official tool for managed cluster-to-cluster replication/migration: mongosync
// mongosync --cluster0 "mongodb://source" --cluster1 "mongodb://target"

In MongoDB, Change Streams are used for CDC: db.collection.watch() streams insert/update/delete events live from the oplog, to be applied to the target (a replica set is required). For full cluster-to-cluster continuous migration/replication, MongoDB's official Cluster-to-Cluster Sync tool, mongosync (not Atlas-specific; it also works with self-managed clusters), manages the initial load + continuous synchronization.

Redis

# Make the target instance a replica of the source (live synchronization)
# On the target Redis:
REPLICAOF source_host 6379
# Monitor sync status:
INFO replication   # master_link_status must be up

# Once the cutover is complete, make the target standalone:
REPLICAOF NO ONE

In Redis, a zero-downtime cutover is done with native replication: run REPLICAOF <source host port> (formerly SLAVEOF) on the target server; the target first receives the full RDB, then stays continuously in sync via the command stream. Verify master_link_status:up with INFO replication, and once the cutover is complete, make the target standalone with REPLICAOF NO ONE.

Elasticsearch

# Incremental reindex from the old cluster to the new one within the same version family
POST _reindex?wait_for_completion=false
{ "source": { "remote": { "host": "http://old:9200" }, "index": "customer",
    "query": { "range": { "updated_at": { "gt": "<last-cursor>" } } } },
  "dest": { "index": "customer" } }

# Licensed feature: Cross-Cluster Replication (CCR) — continuous following
# On the target (follower) cluster: PUT /customer/_ccr/follow { "remote_cluster": "old", "leader_index": "customer" }

The real continuous replication feature in Elasticsearch is Cross-Cluster Replication (CCR): the target cluster follows the leader index on the source and pulls changes continuously (requires a commercial/Platinum license). Without a license, near-continuous sync is achieved with periodic remote reindex (incremental, based on a time/sequence field).

Open in DBAtlas →