1. The Headline

Millions of concurrent users, massive 100,000-member servers, and sub-millisecond voice latency.

Discord started as a voice chat app for gamers. To succeed, they had to beat Skype and TeamSpeak by offering seamless in-browser voice, massive persistent text channels (Guilds), and an architecture that didn't consume all of a gamer's CPU while playing.

2. Requirements and Constraints

Functional Requirements:

• Persistent text chat in channels grouped by "Guilds" (servers).
• Drop-in/drop-out low-latency voice channels.
• Presence tracking (knowing who is online, what game they are playing, and typing indicators).

Non-Functional Requirements:

• Massive Fanout: If a user types a message in a 100,000-member server, 100,000 clients need to receive a WebSocket push instantly.
• Low Resource Usage: The desktop/browser client must be lightweight so it doesn't drop the user's game framerate.

The Ultimate Constraint: The "Megaserver" problem. Most chat apps (like early Slack) loaded the entire team's state on boot. When Discord communities (like the Fortnite or Midjourney servers) grew to millions of members, sending the entire member list and presence state to every client on boot would melt both the backend database and the frontend browser memory.

3. The Naive Design & Where It Breaks

A naive approach to a chat application:

• Client connects via WebSocket.
• Server queries the database: SELECT * FROM users WHERE guild_id = X.
• Server sends the massive JSON payload of all members and their online status to the client.
• When someone comes online, broadcast an event to everyone in the guild.

Where this breaks under Discord's load:

1. The Thundering Herd: If a backend server restarts and 100,000 WebSockets suddenly reconnect simultaneously, the resulting SELECT queries to build their initial state will instantly take down the database.
2. OOM Crashes: Loading a JSON array of 1 million users into a Chrome tab will cause the browser to run out of memory and crash.
3. Event Fanout: Broadcasting a "User X is typing" event to a 1,000,000-member server generates 1,000,000 WebSocket messages for a completely ephemeral event, saturating network bandwidth.

4. The Real Architecture: Layer by Layer

The Gateway (WebSockets)#

Discord handles connections via a tier of Gateway servers (written in Elixir/Erlang).

• Erlang is designed for massive concurrency. Millions of WebSockets terminate at these Gateways.
• The Gateway's only job is holding the connection open and routing messages. It contains very little business logic.

Guild Rings & Pub/Sub#

Behind the Gateway, Discord uses a massive Pub/Sub event bus.

• When a user sends a message to the general channel, the request hits an API server.
• The API saves the message to a fast NoSQL database (originally Cassandra, later ScyllaDB) which handles billions of writes efficiently.
• The API publishes the message to the event bus. The Gateway servers subscribed to that guild pull the message and push it down the specific WebSockets of users who are currently viewing that channel.

Lazy Loading and Presence#

To solve the "Megaserver" problem, Discord completely changed how state is synced:

• Offline Members: Discord stops sending offline members to the client once a server reaches 1,000 members.
• Viewport Subscriptions: The client only subscribes to the presence updates of people currently visible on the screen. If you scroll down the member list, the frontend asks the server for the next chunk of users dynamically.

5. The Hard Problem

The "Spike" of Reconnects.

When a Discord Gateway server crashes, millions of clients immediately try to reconnect. If they all ask for their initial state (unread messages, guild lists) at the same time, it causes a cascading failure across the infrastructure.

6. What This Means for the Client (Frontend)

The Discord frontend (React/Electron) is heavily optimized to protect the backend from reconnect spikes.

Session Resumption#

When the frontend connects to the Gateway, it receives a session_id and a seq (sequence) number representing the last event it received. If the WebSocket drops, the client does not ask for a full state sync when it reconnects. It sends a RESUME payload with its seq number. The Gateway simply replays the missed events from an in-memory buffer, bypassing the database entirely.

Exponential Backoff with Jitter#

If the server is truly down and rejects the RESUME, the client must reconnect from scratch. To prevent a DDoS attack of reconnects, the frontend implements Exponential Backoff (wait 1s, then 2s, 4s, 8s). Crucially, it adds Jitter (randomizing the wait time slightly, e.g., 4.3s instead of exactly 4.0s). This spreads out the reconnect attempts across thousands of clients, smoothing the traffic spike.

Virtualized Lists#

Because guilds can have thousands of active users, the frontend uses DOM Virtualization (like react-window). It only renders the <div> elements for the users currently visible in the sidebar scroll area, recycling DOM nodes as you scroll to keep memory usage minimal.

7. Failure Modes & Graceful Degradation

• Message Send Failure: If Cassandra is struggling and the API times out, the client shows the message in red text with a "Retry" button, rather than silently dropping it.
• Voice Fallback: Voice traffic uses UDP (WebRTC) for extreme low latency. If UDP is blocked by a strict corporate firewall, the client gracefully falls back to TCP.

8. Numbers & Tradeoffs

• Database: Migrated from MongoDB to Cassandra, and eventually to ScyllaDB to handle trillions of messages without garbage collection pauses.
• Tradeoff: Cassandra/ScyllaDB provides incredible write speed and horizontal scalability, but sacrifices complex relational queries. Discord cannot easily do JOIN queries across messages and users; they have to denormalize the data heavily at write time.

9. How to Use This in an Interview

If an interviewer asks you to design a massive chat system like Discord or Slack:

"For the WebSocket layer, we must separate the connection-holding Gateways from the business logic APIs. To handle millions of concurrent users, we cannot send full state on connection. The frontend must heavily lazy-load state, only subscribing to presence updates for users currently in the viewport."

"To protect the backend during outages, the frontend must implement Session Resumption to replay missed events, and use Exponential Backoff with Jitter for reconnects to prevent thundering herds."

10. Sources

• How Discord Stores Billions of Messages
  https://discord.com/blog/how-discord-stores-billions-of-messages
• How Discord handles Two and Half Million Concurrent Voice Users using WebRTC
  https://discord.com/blog/how-discord-handles-two-and-half-million-concurrent-voice-users-using-webrtc