Real-time features in HLD - Scalability & System Analysis

Scalability Analysis - Real-time features

Growth Table: Real-time Features Scaling

Users	Connections	Message Rate	Latency	Infrastructure Changes
100	~100 concurrent	Low (few msgs/sec)	<100ms	Single server with WebSocket support
10,000	~10,000 concurrent	Moderate (hundreds msgs/sec)	<200ms	Load balancer + multiple app servers + Redis pub/sub
1,000,000	~1M concurrent	High (thousands msgs/sec)	<300ms	Clustered message brokers (Kafka, Redis Cluster), sharded app servers, CDN for static content
100,000,000	~100M concurrent	Very High (millions msgs/sec)	<500ms	Global distributed clusters, edge computing, advanced partitioning, multi-region data centers

First Bottleneck

The first bottleneck is the application server's ability to maintain concurrent connections. Real-time features rely on persistent connections like WebSockets, which consume server memory and CPU. Around 5,000 concurrent connections per server is typical. Beyond this, servers struggle to keep connections alive and process messages quickly.

Scaling Solutions

• Horizontal scaling: Add more app servers behind a load balancer to distribute connections.
• Message brokers: Use systems like Redis Pub/Sub, Kafka, or MQTT brokers to handle message distribution efficiently.
• Caching: Cache frequent data to reduce backend load.
• Sharding: Partition users or channels across servers to limit connection and message load per server.
• CDN and edge computing: Offload static content and some processing closer to users to reduce latency and bandwidth.
• Connection multiplexing: Use protocols like HTTP/2 or WebTransport to optimize connection usage.

Back-of-Envelope Cost Analysis

• At 10,000 users with 1 message per second: 10,000 messages/sec to handle.
• Each message ~1KB -> 10MB/s bandwidth needed.
• Storage depends on message retention; 1 day of messages at 10,000 msgs/sec = ~864GB.
• Network bandwidth per server limited to ~1Gbps (~125MB/s), so multiple servers needed.
• CPU and memory scale with connection count; 1 server ~5,000 connections.

Interview Tip

Start by defining the real-time feature and expected load. Identify the main challenges: connection management, message throughput, and latency. Discuss bottlenecks in servers and network. Propose scaling steps: horizontal scaling, message brokers, caching, and sharding. Always mention trade-offs and monitoring needs.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Introduce read replicas and caching layers to reduce load on the primary database before scaling vertically or sharding.

Key Result

Real-time features first hit limits on concurrent connections at app servers; horizontal scaling and message brokers are key to scaling beyond 10K users.

Practice

1. Which protocol is commonly used to enable real-time communication in web applications?

easy

A. SMTP

B. HTTP/1.1

C. WebSocket

D. FTP

Solution

1. Step 1: Understand real-time communication needs

   Real-time apps require a protocol that supports two-way, instant data exchange.

2. Step 2: Identify protocol features

   WebSocket allows full-duplex communication over a single connection, unlike HTTP/1.1 which is request-response only.

3. Final Answer:

   WebSocket -> Option C

4. Quick Check:

   Real-time = WebSocket [OK]

Hint: Real-time needs two-way instant data flow: WebSocket fits best [OK]

Common Mistakes:

• Confusing HTTP with WebSocket for real-time
• Choosing FTP or SMTP which are not real-time protocols
• Thinking HTTP/2 is the same as WebSocket

2. Which component in a real-time system is responsible for distributing messages from producers to consumers?

easy

A. Broker

B. Producer

C. Consumer

D. Database

Solution

1. Step 1: Define roles in real-time messaging

   Producers send data, consumers receive data, and brokers route messages between them.

2. Step 2: Identify the distributor

   The broker acts as the middleman ensuring messages reach the right consumers.

3. Final Answer:

   Broker -> Option A

4. Quick Check:

   Message routing = Broker [OK]

Hint: Broker connects producers and consumers by routing messages [OK]

Common Mistakes:

• Confusing producer as distributor
• Thinking consumer sends messages
• Assuming database handles message routing

3. Consider a chat app using WebSocket. If the server receives a message from User A and broadcasts it to 100 connected users, what is the main bottleneck to scale this real-time feature?

medium

A. User A's device speed

B. Client browser rendering speed

C. Database read latency

D. Server CPU and network bandwidth

Solution

1. Step 1: Analyze message flow in real-time chat

   The server receives and then sends the message to all connected users, requiring CPU and network resources.

2. Step 2: Identify bottleneck

   Server CPU handles message processing; network bandwidth handles sending to many users simultaneously.

3. Final Answer:

   Server CPU and network bandwidth -> Option D

4. Quick Check:

   Scaling real-time = Server resources [OK]

Hint: Server resources limit broadcast scale, not client or DB speed [OK]

Common Mistakes:

• Blaming client device speed for server load
• Focusing on database latency which is less critical here
• Ignoring network bandwidth limits

4. A real-time notification system uses MQTT but users report delayed messages. Which is the most likely cause?

medium

A. Clients are using WebSocket instead of MQTT

B. Broker is overloaded and dropping messages

C. Messages are too small to send quickly

D. Users have disabled notifications on their devices

Solution

1. Step 1: Understand MQTT broker role

   The broker routes messages; if overloaded, it queues or drops messages causing delays.

2. Step 2: Evaluate other options

   Clients using WebSocket instead of MQTT would cause connection issues, not delays; small messages send faster; disabled notifications affect display, not delivery.

3. Final Answer:

   Broker is overloaded and dropping messages -> Option B

4. Quick Check:

   Delays usually mean broker overload [OK]

Hint: Delays often mean broker overload, not client protocol mismatch [OK]

Common Mistakes:

• Blaming client protocol mismatch for delays
• Assuming small messages cause delays
• Ignoring broker capacity limits

5. You are designing a real-time stock price update system for millions of users. Which approach best ensures scalability and low latency?

hard

A. Use a distributed message broker cluster with topic partitions and WebSocket connections

B. Store all prices in a single database and poll clients every second

C. Send updates via email to all users when prices change

D. Use HTTP long polling from clients to server for updates

Solution

1. Step 1: Understand scalability needs

   Millions of users require distributed systems to handle load and maintain low latency.

2. Step 2: Evaluate options

   Distributed brokers with topic partitions allow parallel processing; WebSocket supports instant push updates. Polling and email cause delays and high load.

3. Final Answer:

   Use a distributed message broker cluster with topic partitions and WebSocket connections -> Option A

4. Quick Check:

   Scalable real-time = Distributed broker + WebSocket [OK]

Hint: Distributed brokers + WebSocket scale best for millions [OK]

Common Mistakes:

• Choosing polling which wastes resources and adds latency
• Using email which is not real-time
• Relying on a single database causing bottlenecks