DBAtlas · Database operations atlas

Rolling maintenance on a replica set (restart, OS patch, minor upgrade)

MongoDBReplica Set

Command

// 1) Before starting: all members healthy, low lag, and an oplog window longer than the maintenance
rs.printSecondaryReplicationInfo()
// 2) Clean shutdown on the SECONDARY under maintenance (then OS/binary work, restart with the same config):
db.getSiblingDB("admin").shutdownServer({ timeoutSecs: 60 })
// 3) Wait until the member is SECONDARY and lag has closed, then move to the next secondary:
rs.status().members.map(m => ({ host: m.name, state: m.stateStr, health: m.health }))
// 4) When all secondaries are done, on the PRIMARY: hand over to a secondary that catches up within 30 s
rs.stepDown(120, 30)
// 5) The old primary is now a SECONDARY: apply steps 2 and 3 to it

What it does

Gives the order for taking members into maintenance one at a time, secondaries first and the primary last, so restarts, OS patches or binary updates are done without taking the set out of service.

When to use it

During planned OS/kernel updates, minor version upgrades, mongod parameter changes or disk/hardware maintenance.

Caution

Never shut down more than one member at a time; in a 3-member set losing two members loses the majority. After each member, confirm SECONDARY state and that lag has closed. stepDown causes a short write interruption; make sure drivers reconnect with retryable writes. If there is an arbiter, handle it in the same sequence.

Open in DBAtlas →

Related