Cassandra 5 gives you real reasons to upgrade: the unified compaction strategy (UCS), storage-attached indexes (SAI), native vector types, and trie-based memtables. None of them matter if the upgrade takes down reads on the way. Major-version upgrades in Cassandra are well-trodden and safe — when run with ceremony. This is the ceremony we use.
Before anything: the pre-flight audit
Version path. Cassandra 5 upgrades from the 4.x line. A cluster still on 3.x goes through 4.x first — budget it as two upgrades, not one.
Deprecated and removed settings. Diff your cassandra.yaml against the new defaults. Cassandra 4.1+ renamed a swath of settings into the new duration/size formats; 5 removes more. A node that won't start because of an unrecognized setting is the most preventable upgrade failure there is — catch it in staging, not on node one.
Driver compatibility. Confirm client driver versions against the target server version before touching servers, and check protocol-version pinning: a client that negotiated a fixed protocol version needs review, not hope.
Cluster health. No upgrade starts on a sick cluster: nodetool status all UN, repairs current on every table, no accumulating hints, disk headroom for upgradesstables (assume you need free space comparable to the largest table's footprint per node).
Snapshots. nodetool snapshot cluster-wide, and verify the snapshots exist and are restorable in principle. This is your true rollback for the data files.
Staging rehearsal. Run the whole procedure on a staging cluster restored from production snapshots, with production-shaped load replayed against it. Every surprise you meet in staging is one you skip in production.
The rolling upgrade, node by node
The procedure is deliberately boring:
- Confirm quorum math: with RF=3 and
LOCAL_QUORUM, one node down per rack at a time is safe. Upgrade one node at a time anyway unless the cluster is large and rack-aware automation is proven. - On the target node:
nodetool drain(flushes memtables, stops accepting writes cleanly), stop the service. - Upgrade the package/binaries; apply the reviewed configuration.
- Start; watch the log to
state jump to NORMAL; confirmnodetool statusshows UN cluster-wide and clients see the node. - Let it soak. Ten quiet minutes of dashboards — read/write latency, dropped messages, exceptions — before the next node. The pause is the safety mechanism.
- Repeat, one node at a time, rack by rack.
Mixed-version rules while the roll is in progress: no schema changes (DDL), no nodetool repair, no topology changes (no bootstrapping or decommissioning nodes). Mixed-version clusters serve traffic fine; they do not handle schema agreement or streaming reliably. Keep the mixed-version window short — hours or days, not weeks.
After the last node: upgradesstables
SSTables remain in the old format until rewritten. The cluster works without rewriting — Cassandra reads old formats — but new features and full performance arrive with the new format, and lingering old-format files complicate future upgrades. Run:
nodetool upgradesstables
node by node, throttled, off-peak. It is a full rewrite of affected data and competes with compaction for I/O; treat it as a scheduled background campaign over days, not a step to rush the same evening. Only after it completes everywhere do we call the upgrade done — and only then do we resume repairs and consider schema changes.
Rollback criteria, agreed in advance
Write them down before node one: what metric regression (e.g. p99 read latency +50% sustained 15 minutes, dropped mutations non-zero and climbing) triggers a pause, and what triggers rollback. Rolling back a partially upgraded cluster is straightforward for nodes that haven't run upgradesstables (stop, reinstall old version, restart — the snapshots cover the worst case); after SSTables are rewritten, rollback means restore-from-snapshot. This is exactly why upgradesstables waits until the whole cluster has soaked on the new version.
What to do with 5 once you're on it
Don't flip every new feature at once. Our order: let the cluster run vanilla for a week; then evaluate UCS per table (it genuinely simplifies the STCS/LCS/TWCS decision for mixed workloads, but migrate compaction strategy one table at a time, watching backlog); then SAI where you've been paying for old secondary indexes; vector types when the application is ready for them.
An upgrade run this way is anticlimactic. That is the point — and if you'd rather have someone alongside who has run the ceremony before, that's precisely the shape of work we do by the hour.