MongoDB rolling upgrade (7.0 → 8.0): replica set and sharded cluster

Upgrades a running MongoDB 7.0 replica set or sharded cluster to 8.0 without downtime (rolling): pre-checks (version, FCV, member state, driver compatibility), backup, secondaries first, then stepDown and the former primary; for a sharded cluster the order is stop the balancer, config servers, shards, mongos. FCV 8.0 can optionally be deferred until after the burn-in period; rollback options and their limits are included. For a patch update within the same major version (8.0.x → 8.0.y) the repo stays the same and the same member order is followed; coming from 6.0, upgrade to 7.0 first.

Topology: Replica set (3+ members) or sharded cluster (config servers + shards + mongos)

Before you start

• All …mongod instances (config server and shard members) and mongos…members… run 7.0.x with FCV 7.0. If you are coming from 6.0, upgrade to 7.0 first (a major version cannot be skipped).
• No member is in ROLLBACK/RECOVERING/STARTUP2 state; the secondaries are a few seconds behind the primary.
• The oplog window is much longer than the maintenance time (downtime per member), at least several hours. If it is short, the member that was down needs an initial sync when it comes back.
• All application drivers are on a version compatible with MongoDB 8.0 (driver compatibility table). Old drivers may fail to connect to 8.0 or may not support the new behavior.
• The 8.0 compatibility changes page has been read; you do not use any removed or changed feature (parameter, command, query behavior). If possible, the upgrade was rehearsed in a test environment first.
• A current backup with a tested restore exists (snapshot or mongodump). The maintenance window is approved.
• Every member has enough free space on the data disk and in /var/log, and has access to the 8.0 repo (repo.mongodb.org).
• On the config server and shard members mongod runs with the mongod systemd service and /etc/mongod.conf; on the mongos servers the service name is …. Each server runs a single mongod/mongos process.
• The balancer can be stopped during the upgrade window; no chunk migration, resharding or index build is in progress. No shard key or collection metadata change is planned during the maintenance.

Phases of the guide

• Pre-checks — Version, FCV and role (all …mongod instances…members…) · Member state and replication lag · Cluster state, shards and balancer · Invalid system.buckets collections · Driver and compatibility check · Backup and fallback point …
• Repo change (all members) — Add the 8.0 repo, disable the 7.0 repo · Add the 8.0 repo, disable the 7.0 repo
• Upgrade the secondaries one at a time — Secondary: verify that … is not primary and stop it · Secondary: install the 8.0 binaries on … · Secondary: configure THP for 8.0 on … · Secondary: start … and wait for SECONDARY · Secondary: verify that … is not primary and stop it · Secondary: install the 8.0 binaries on … …
• Step down and upgrade the primary — Verify that the secondaries are on 8.0 and healthy · Step down the primary (stepDown) · Former primary: verify that … is not primary and stop it · Former primary: install the 8.0 binaries on … · Former primary: configure THP for 8.0 on … · Former primary: start … and wait for SECONDARY
• Stop the balancer — Stop the balancer and verify that migrations have finished
• Start the balancer and verify the versions — Verify that all components are on 8.0 · Start the balancer again
• Verification — All members on 8.0 · Application and log check
• featureCompatibilityVersion 8.0 — Raise FCV to 8.0 · Verify FCV
• Rollback (only if there is a problem) — Choose the rollback path · Roll back the binaries to 7.0: … · Restore from backup · Rollback order in a sharded cluster