Concept
Every topic so far in this domain has assumed the request-response model: a client sends a request, a server sends back a response, the connection can be treated as effectively stateless and short-lived. Real-time systems, live chat, collaborative editing, live dashboards, multiplayer features, break this assumption at the root: a WebSocket connection is long-lived, and the server holding it needs to be able to push a message to that specific client at any time, not just in response to a request that client just made.
Why this breaks the horizontal-scaling model from earlier topics
Recall from the Scalability topic: a load balancer distributes independent, short-lived requests across interchangeable stateless servers, and any server can handle any request. WebSockets violate the "interchangeable" part directly. Once a client establishes a WebSocket connection, that connection is pinned to whichever specific server instance accepted it, the TCP connection itself lives on that one machine. If another user sends a chat message that needs to reach the first user, and the first user's connection happens to be held by server B while the message originated on server A, server A has no way to deliver it directly, it doesn't have that TCP connection.
User A ──WebSocket──▶ Server 1 (holds A's connection)
User B ──WebSocket──▶ Server 2 (holds B's connection)
User A sends a chat message meant for User B.
Server 1 receives it, but Server 1 does NOT have User B's
connection. It cannot just "send" the message to B directly.The fix: pub/sub as the cross-server delivery mechanism
This is exactly where the Message Queues & Pub/Sub topic's fan-out mechanism becomes essential, not optional. The standard architecture: every server instance subscribes to a shared pub/sub channel (commonly Redis Pub/Sub, or a dedicated service built for this). When a message needs to be delivered, the receiving server publishes it to the shared channel instead of trying to deliver it directly, every other server instance, including whichever one actually holds the target recipient's connection, receives that published message and checks: "do I have a connection for this recipient? If so, push it down that connection."
User A ──WebSocket──▶ Server 1 ──publish──▶ [shared pub/sub channel]
│
┌────────────────────────────┼────────────────────────────┐
▼ ▼ ▼
Server 1 Server 2 Server 3
(checks: do I have B's (checks: do I have B's (checks: do I have B's
connection? No.) connection? YES → connection? No.)
pushes down B's socket)
User B ──WebSocket──▶ Server 2 ← message deliveredThis is precisely the earlier "point-to-point queue vs. pub/sub" distinction paying off in a concrete architecture: every server instance needs its own copy of every published message (to check its own local connection table), which is fan-out, pub/sub's defining behavior, not point-to-point delivery.
Horizontal scaling of WebSocket servers: what changes and what doesn't
The load balancer still distributes new incoming connections across server instances (often using an algorithm like least-connections, since WebSocket connection counts, not request counts, are what's being balanced). What changes is that once a connection is established, it stays pinned to that one server for its entire lifetime, there's no "round robin the next message from this same connection to a different server," because it's the same TCP connection the whole time. Scaling a WebSocket tier horizontally means adding more server instances to hold more concurrent connections (each instance has a practical ceiling on how many open sockets it can hold), combined with the pub/sub layer so any server can still reach any connection anywhere in the fleet, the two techniques (add instances, wire them together with pub/sub) are both necessary; neither alone is sufficient.
The fallback ladder: not every client can hold a persistent connection
Real systems can't assume a WebSocket connection is always available, some corporate proxies and older infrastructure block or don't support them well. The standard fallback ladder, roughly in order of preference:
1. WebSocket: genuine persistent, full-duplex connection. Lowest
latency, lowest overhead per message. Preferred
when available.
2. Long polling: client sends a request; server HOLDS it open
(doesn't respond immediately) until there's actually
new data to send, THEN responds, client immediately
re-issues a new request. Simulates push over plain
HTTP, at the cost of one HTTP request per "message"
and the server needing to hold many open requests.
3. Short polling: client just asks "anything new?" every N seconds
on a fixed interval, regardless of whether there's
actually anything new. Simplest, but wastes requests
when nothing has changed and has up-to-N-seconds
latency for genuinely new data.A production real-time system typically attempts WebSocket first, and transparently falls back down this ladder if the connection can't be established or drops unexpectedly, libraries like Socket.IO implement exactly this negotiation automatically, so application code can mostly ignore which transport is actually in use.
Try It
A live chat app has 3 WebSocket server instances behind a load balancer, no pub/sub layer wired up yet, and a "typing indicator" feature where User A's client should see "User B is typing..." in real time. User A is connected to Server 1, User B is connected to Server 2. What happens when User B starts typing, and what's the minimal fix?