GoKawiilRaft Consensus with a Minority of Nodes
tl;dr — This post describes a (wacky) modification to the Raft consensus protocol such that progress can be made even if fewer than a majority of nodes are actively participating, given some constraints on exactly which minority of nodes are active. The math behind this comes from the same place as the card game Spot It! (Dobble).
Raft Consensus Basics
Raft is a consensus protocol for managing a replicated log across a cluster of nodes. Its key goals are: (1) maintain a consistent replicated log of state transitions, (2) tolerate node failures, and (3) ensure a single leader coordinates all changes while multiple followers replicate. Raft is designed to be understandable — it decomposes consensus into leader election, log replication, and safety — and is widely used in systems like etcd, CockroachDB, and TiKV.
In steady state, the leader receives client requests and appends them to its log. It then sends AppendEntries RPCs to all followers. Once a majority of nodes (including itself) have appended the entry, the leader considers it "committed" and applies it to the state machine. For example, in a 5-node cluster, the leader needs acknowledgments from at least 2 followers (3 total including itself) before committing. This provides fault tolerance for up two node failures or a network partition where at least a majority of nodes are able to communicative with each other.
If the leader crashes, a new one is elected. Any node can become a candidate, start an election for a new "term," and request votes. A candidate wins if it receives votes from a majority of nodes. This guarantees that at most one leader exists per term. Once elected, the new leader synchronizes followers' logs and resumes normal operation.
The key correctness insight is this: any two majorities of nodes must overlap in at least one node. So between any two consecutive global state changes — whether two commits, two leader elections, or one of each — at least one node participated in both. This single overlapping node carries forward the knowledge of what was previously committed, preventing conflicts and ensuring consistency. In a 5-node cluster, any two sets of 3 nodes must share at least one member. This overlap is what makes Raft safe.
Spot It!
Spot It! (also known as Dobble) — the author's favorite family card game.
Spot It! (known as Dobble outside North America) is a card game whose rules are relatively straightforward: flip a card from the deck to the center, and race to find the one symbol your card has in common with the center card. Call it out, discard your card, and repeat. It's fast, fun, and requires no reading or arithmetic. It's simple enough for a 5-year old to learn quickly, yet the game design is surprisingly complex.
... continue reading