1. The Headline

300+ million daily meeting participants with sub-150ms latency.

When the pandemic hit, Zoom dominated the video conferencing market precisely because it just worked, even on terrible internet connections. Achieving this required abandoning traditional peer-to-peer WebRTC and building a globally distributed, custom-routed video architecture designed entirely around latency mitigation.

2. Requirements and Constraints

Functional Requirements:

• Many-to-many video and audio chat (up to 1,000 participants).
• Screen sharing with high resolution.
• Dynamic layout changes (Active Speaker vs. Gallery View).

Non-Functional Requirements:

• Latency: Must remain under 150ms to prevent people from talking over each other.
• CPU Efficiency: Decoding 49 video streams simultaneously cannot melt the user's laptop.
• Network Resilience: Must handle extreme packet loss gracefully without dropping the call.

The Ultimate Constraint: Upstream bandwidth on home internet connections. Most users have asymmetric internet (e.g., 100Mbps down, but only 10Mbps up). A user cannot upload 49 separate video streams to 49 different participants in a large meeting.

3. The Naive Design & Where It Breaks

A naive approach to video chat: Peer-to-Peer (P2P) Mesh.

• Alice, Bob, and Charlie join a call.
• Alice sends her video stream directly to Bob, and directly to Charlie.
• Bob sends to Alice and Charlie.

Where this breaks:

1. Upstream Exhaustion: In a 10-person meeting, every person has to upload 9 separate 720p streams. Home internet connections will instantly choke, leading to frozen video.
2. CPU Overload: The client has to encode the video 9 times and decode 9 incoming streams, destroying laptop batteries.

Another naive approach: Multipoint Control Unit (MCU).

• Everyone sends 1 stream to a central server.
• The server decodes all the streams, stitches them together into a single "Brady Bunch" 720p video grid, re-encodes it, and sends 1 single stream back to everyone.
• Where this breaks: Decoding and re-encoding video on a central server adds hundreds of milliseconds of latency, making conversation impossible. It is also massively expensive for the server CPU.

4. The Real Architecture: Layer by Layer

The Selective Forwarding Unit (SFU)#

Zoom uses a Selective Forwarding Unit (SFU) architecture.

• Every participant uploads exactly one stream to the central Zoom server (the SFU).
• The SFU does not decode or stitch the video. It is a "dumb" router.
• The SFU simply takes Alice's stream and rapidly forwards the raw packets to Bob, Charlie, and Dave.
• This solves the upstream bandwidth problem (Alice only uploads once) and the server latency problem (the SFU just routes packets instantly without decoding them).

The Multi-Media Routing Network#

Zoom operates its own global overlay network. Instead of trusting the unpredictable public internet to route video packets from Tokyo to New York, the Zoom client routes the video to the nearest Zoom edge server. The video travels across Zoom's private, highly-optimized fiber backbone to the destination edge server, vastly reducing latency and packet loss.

5. The Hard Problem

Heterogeneous downstream connections.

The SFU solves the upstream problem, but what if Alice is broadcasting in 1080p, and Bob is watching on a 5G connection, but Charlie is watching on a terrible 3G connection? If the SFU forwards Alice's 1080p stream to Charlie, Charlie's connection will choke, and he will drop from the call.

6. What This Means for the Client (Frontend)

To solve the heterogeneous connection problem, Zoom utilizes Simulcast and aggressive client-side bandwidth adaptation.

Simulcast#

The Zoom client doesn't actually upload just one stream. It uses Simulcast to upload multiple spatial layers (e.g., one 1080p stream, one 360p stream, and one 180p stream) simultaneously. Because lower resolutions are so tiny, uploading all three barely uses more bandwidth than 1080p alone.

The SFU server looks at the downstream connection of every receiver:

• It forwards the 1080p stream to Bob (good connection).
• It forwards the 180p stream to Charlie (bad connection). If Charlie's connection improves, the SFU seamlessly switches to forwarding the 360p stream without asking Alice to change anything.

Layout-Driven Subscriptions (Gallery vs. Active Speaker)#

The Zoom frontend heavily dictates the backend routing.

• If you are in Active Speaker mode, your client tells the SFU: "Only send me the 1080p stream of the person talking, and drop the video for everyone else." This saves massive downstream bandwidth.
• If you switch to Gallery View (49 people), your client tells the SFU: "Send me the 180p streams of all 49 people." The client then decodes 49 tiny streams and renders them in a CSS Grid. (Decoding 49 tiny streams is far less CPU-intensive than decoding 49 HD streams).

7. Failure Modes & Graceful Degradation

• Packet Loss Mitigation: Zoom heavily favors audio over video. If the connection degrades, the client aggressively drops the video framerate (from 30fps down to 5fps), and eventually kills the video entirely, dedicating all remaining bandwidth to keeping the audio stream alive.
• Forward Error Correction (FEC): The client redundantly encodes extra math into the audio packets. If a packet is lost in transit, the receiving client can use the FEC math to guess the missing audio, preventing the "robot voice" clipping effect.

8. Numbers & Tradeoffs

• Architecture: SFU (Selective Forwarding Unit) with client-side Simulcast.
• Tradeoff: Using an SFU instead of an MCU pushes the decoding burden onto the client device (e.g., rendering 49 videos). Zoom mitigated this by building deep integrations with local GPU hardware acceleration, writing custom C++/WebAssembly decoders to bypass browser bottlenecks.

9. How to Use This in an Interview

If an interviewer asks you to design a video conferencing or live-streaming app:

"A P2P mesh cannot scale beyond a few participants due to upstream bandwidth constraints. An MCU (stitching video on the server) adds too much latency. We must use an SFU (Selective Forwarding Unit) which simply routes packets. To handle users with bad internet, the broadcasting client should use Simulcast to send multiple resolutions, allowing the SFU to dynamically downgrade the stream for struggling receivers."

10. Sources

• Under the Hood of Zoom's Architecture
  https://blog.zoom.us/zoom-architecture/