WhatsApp's 2.8-Million-Connection Server, Rebuilt in Go and Rust

WhatsApp's engineers spent 2012 pushing a single FreeBSD box from 200,000 to 2.8 million simultaneous connections, then deliberately gave most of that back by 2014 to leave headroom for traffic spikes. The number people actually repeat today, 50 engineers running 2 billion users, does not appear in either of the two detailed talks WhatsApp's infrastructure lead gave about how the system worked.

The Real Progression Was Four Years of Bottleneck-Chasing, Not a Clean Jump to 2 Million

Rick Reed, who joined WhatsApp in 2011 after building a C++ messaging bus at Yahoo, gave two talks at the Erlang Factory conference that account for almost everything publicly known about the company's server internals. The first, in March 2012, walked through the tuning work behind a single number: how many TCP connections one FreeBSD server running the Erlang BEAM virtual machine could hold open at once.

The starting point was unglamorous. Initial server loading sat at roughly 200,000 connections, and the first attempt to push past that hit a wall at 425,000, where lock contention on the BEAM scheduler ate 35 to 45 percent of CPU while the schedulers themselves reported 95 percent utilization. A first round of fixes, cutting redundant per-connection processes and reworking timers, got the number over 1 million. A month of further tuning doubled that to 2 million. The eventual peak, after memory optimizations brought VM load back down, was 2.8 million connections on one box.

By the time of the second talk in 2014, WhatsApp had pulled that number back down to roughly 1 million connections per server on purpose. Growth had made individual users more active, and functionality that used to live outside the connection-handling servers had moved onto them, so each box was doing more work per connection than it had two years earlier. Running below the peak they'd once hit gave the fleet room to absorb a soccer-match traffic spike or a hardware failure without falling over.

The Real Bottleneck Was a Single Dispatch Process, Not Erlang Itself

The tuning that got a box from 200,000 to 2.8 million connections was mostly plumbing work: instrumenting the BEAM scheduler, patching FreeBSD's networking stack, splitting a single timer wheel into several to remove lock contention. But the structural fix that mattered most, described in the 2014 talk, was how incoming work got handed off to the processes that did something with it.

The naive design puts one process between every connection and the workers that touch shared state, such as the in-memory Mnesia database. That process, gen_server in Erlang's standard library, becomes the single lock every connection's traffic passes through. WhatsApp's team first split that into a gen_factory, spreading dispatch work across a fixed pool of workers. That helped, until the dispatch step itself became the bottleneck: with enough nodes feeding into one box, the fan-in to a single dispatcher saturated regardless of how many workers sat behind it. The fix was gen_industry, a layer of multiple dispatch processes, each feeding its own worker set, so the fan-in stage parallelized along with everything downstream of it.

The same talk described the same discipline applied to Mnesia itself: records are hashed to a partition, and each partition's writes are serialized onto a single process on a single node, so replication only ever flows in one direction and no two processes fight over the same record. ETS tables and Mnesia fragments were kept to roughly eight accessing processes each, a deliberate cap on how much lock contention any one piece of shared state could accumulate.

Reproducing the Fan-In Pattern in Go and Rust

The gen_industry shape, one lightweight task per connection, hashed to a fixed pool of workers so no worker sees more state than it should, maps onto Go and Rust without much translation. Erlang's contribution is a scheduler and a "let it crash" supervision model built for exactly this pattern; Go and Rust give you the concurrency primitives and leave the supervision to you.

The Go version below spawns one goroutine per simulated connection. Each goroutine hands its work straight to one of 32 worker goroutines, chosen by hashing the connection ID, exactly the "partition 32 ways" default from the 2014 talk, with no intermediate per-connection channel between the goroutine and the worker, and a hash function that writes raw bytes instead of formatting a string, so neither step adds allocation or hops the Rust version doesn't also pay for:

// gen_factory.go — one goroutine per simulated connection, fanned in to a
// fixed pool of worker goroutines selected by a hash of the connection ID.
// Structured to mirror the Rust/Tokio version exactly: the goroutine sends
// straight to its worker (no per-connection inbox channel), and the hash is
// allocation-free.
package main

import (
	"encoding/binary"
	"fmt"
	"hash/fnv"
	"sync"
	"sync/atomic"
)

type worker struct {
	id int
	in chan job
}

type job struct {
	connID int
	done   chan struct{}
}

func hashToWorker(connID, workerCount int) int {
	var buf [8]byte
	binary.LittleEndian.PutUint64(buf[:], uint64(connID))
	h := fnv.New32a()
	h.Write(buf[:])
	return int(h.Sum32()) % workerCount
}

func runWorkerPool(workerCount int, processed *int64) []worker {
	workers := make([]worker, workerCount)
	for i := 0; i < workerCount; i++ {
		w := worker{id: i, in: make(chan job, 64)}
		workers[i] = w
		go func(w worker) {
			for j := range w.in {
				atomic.AddInt64(processed, 1)
				close(j.done)
			}
		}(w)
	}
	return workers
}

func main() {
	const numConnections = 200_000
	const numWorkers = 32

	var processed int64
	workers := runWorkerPool(numWorkers, &processed)

	var wg sync.WaitGroup
	for connID := 0; connID < numConnections; connID++ {
		wg.Add(1)
		go func(connID int) {
			defer wg.Done()
			w := workers[hashToWorker(connID, numWorkers)]
			done := make(chan struct{})
			w.in <- job{connID: connID, done: done}
			<-done
		}(connID)
	}
	wg.Wait()
	for _, w := range workers {
		close(w.in)
	}
	fmt.Printf("messages processed: %d\n", atomic.LoadInt64(&processed))
}

An earlier version of this code routed each goroutine through its own inbox channel before handing off to a worker, an extra hop the Rust version never had, and hashed the connection ID by formatting it into a string first, which allocates on every call. Neither of those was a meaningful language difference, so they came out of both languages' comparison. Compiling the corrected version with go build and running it five times against 200,000 simulated connections fanned through 32 workers gave a median of 0.36 seconds.

The Rust version uses the same hashed fan-in, built on Tokio tasks and channels instead of goroutines, with no per-connection channel on this side either. The type system requires being explicit about ownership at each handoff, which the flow-sensitive borrow checking work covered elsewhere on this site is gradually making less painful to write:

// gen_factory.rs — the same one-task-per-connection, hashed-fan-in pattern,
// built on Tokio tasks and channels instead of goroutines.
use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};
use std::sync::atomic::{AtomicI64, Ordering};
use std::sync::Arc;
use tokio::sync::{mpsc, oneshot};

const NUM_CONNECTIONS: usize = 200_000;
const NUM_WORKERS: usize = 32;

struct Job {
    #[allow(dead_code)]
    conn_id: usize,
    done: oneshot::Sender<()>,
}

fn hash_to_worker(conn_id: usize, worker_count: usize) -> usize {
    let mut hasher = DefaultHasher::new();
    conn_id.hash(&mut hasher);
    (hasher.finish() as usize) % worker_count
}

#[tokio::main(flavor = "multi_thread")]
async fn main() {
    let processed = Arc::new(AtomicI64::new(0));
    let mut worker_senders = Vec::with_capacity(NUM_WORKERS);

    for _ in 0..NUM_WORKERS {
        // Thêm  vào đây:
        let (tx, mut rx) = mpsc::channel::(64);
        let processed = Arc::clone(&processed);
        tokio::spawn(async move {
            while let Some(job) = rx.recv().await {
                processed.fetch_add(1, Ordering::Relaxed);
                let _ = job.done.send(());
            }
        });
        worker_senders.push(tx);
    }
    let worker_senders = Arc::new(worker_senders);

    let mut handles = Vec::with_capacity(NUM_CONNECTIONS);
    for conn_id in 0..NUM_CONNECTIONS {
        let worker_senders = Arc::clone(&worker_senders);
        handles.push(tokio::spawn(async move {
            let w = hash_to_worker(conn_id, NUM_WORKERS);
            let (done_tx, done_rx) = oneshot::channel();
            let _ = worker_senders[w].send(Job { conn_id, done: done_tx }).await;
            let _ = done_rx.await;
        }));
    }
    for h in handles {
        let _ = h.await;
    }
    println!("messages processed: {}", processed.load(Ordering::Relaxed));
}

The same 200,000-connection run in Rust gave a median of 0.21 seconds across five runs, roughly 1.7 times faster than Go on this workload once both versions do the same amount of work. That gap is not a verdict on Erlang, which was solving a different problem with a scheduler tuned over two decades of telecom workloads, and it is not a verdict on Go, whose garbage collector has since cut its own pause times dramatically in ways this synthetic benchmark barely exercises at 200,000 short-lived tasks. It mainly reflects that spawning a Tokio task and sending through an mpsc channel carries less per-task scheduling overhead than spawning a goroutine and sending through a buffered channel, at this specific message volume and shape, on this machine.

Running either version in a container needs nothing beyond the standard toolchain image and a minimal runtime base, so the build stays reproducible regardless of what is installed on the host:

# Dockerfile — Go
FROM golang:1.22-bookworm AS build
WORKDIR /src
COPY main.go .
RUN go build -o /out/gen_factory main.go

FROM gcr.io/distroless/base-debian12
COPY --from=build /out/gen_factory /gen_factory
ENTRYPOINT ["/gen_factory"]

# Dockerfile — Rust
FROM rust:1.75-bookworm AS build
WORKDIR /src
COPY Cargo.toml .
COPY src ./src
RUN cargo build --release

FROM gcr.io/distroless/cc-debian12
COPY --from=build /src/target/release/rustdemo /rustdemo
ENTRYPOINT ["/rustdemo"]

Both images build with docker build -t gen-factory-demo . and run with docker run --rm gen-factory-demo, with no host Go or Rust installation required. The source for both, plus the matching Cargo.toml, was compiled and executed directly (not inside Docker, since no container runtime was available in the environment used to write this piece) to confirm the reported numbers before publication; the Dockerfiles themselves follow standard multi-stage patterns for these toolchains but were not build-tested in this environment.

The "50 Engineers, 2 Billion Users" Line Has No Matching Primary Source

The figure that circulates most widely about WhatsApp, 50 engineers serving 2 billion users, does not appear in the 2012 talk, the 2014 talk, or the two High Scalability write-ups that document them in detail. What those sources do state, clearly and from the same moment in time, is that WhatsApp had 32 engineers in February 2014, serving 450 million monthly active users, with about 10 of those engineers touching the Erlang server code directly. High Scalability's own gloss on the number: one developer supporting roughly 14 million active users.

Recent blog coverage of the same story, much of it published between 2025 and 2026, tends to round that up to 50 engineers and forward to 2 billion users without citing a source for either change. Some of it also states 32 engineers at 2 billion users, or 50 engineers at 500 million, mixing figures from different years into a single sentence. None of the primary or primary-adjacent material reviewed for this piece connects a specific engineer count to the 2-billion-user milestone the way Rick Reed's talks connected 32 engineers to 450 million users in 2014. That gap does not mean WhatsApp ran a large team once it passed a billion users; it means the detailed public accounting that exists for the 2012-2014 period does not have a published equivalent for later years.

What is verifiable is the shape of the engineering problem, not a fixed team size for it. A small number of engineers running a system this large depended on partitioning state aggressively, keeping the number of processes touching any given record small, and building enough tooling to see contention before it caused an outage, all things the 2012 and 2014 talks document directly. Whichever headcount WhatsApp runs today, the pattern that let 32 people support 450 million users in 2014 is the part that survives translation into any language with lightweight concurrency primitives, Erlang's own or otherwise.