Media sharing in messages in HLD - Scalability & System Analysis

Scalability Analysis - Media sharing in messages

Growth Table: Media Sharing in Messages

Scale	Users	Messages/Day	Media Uploads/Day	Storage Needed	Bandwidth	System Changes
Small	100	10,000	2,000	~20 GB	~50 Mbps	Single server, local storage, simple DB
Medium	10,000	1,000,000	200,000	~2 TB	~500 Mbps	Multiple app servers, CDN for media, DB replicas
Large	1,000,000	100,000,000	20,000,000	~200 TB	~5 Gbps	Sharded DB, distributed storage, multi-CDN, load balancers
Very Large	100,000,000	10,000,000,000	2,000,000,000	~20 PB	~50 Gbps+	Global data centers, advanced sharding, tiered storage, edge caching

First Bottleneck

At small to medium scale, the database becomes the first bottleneck due to high write and read operations for message metadata and media references.

As user count and media volume grow, storage I/O and network bandwidth for media uploads/downloads become bottlenecks.

Eventually, the media storage system and CDN capacity limit scalability before the app servers.

Scaling Solutions

• Database: Use read replicas and connection pooling to handle increased queries.
• Caching: Cache frequently accessed media metadata and message data to reduce DB load.
• Storage: Move media files to distributed object storage (e.g., S3) with lifecycle policies.
• CDN: Use Content Delivery Networks to serve media closer to users, reducing bandwidth and latency.
• Sharding: Partition database and storage by user or message ID to distribute load.
• Horizontal Scaling: Add more app servers behind load balancers to handle concurrent connections.
• Compression & Thumbnails: Store optimized media versions to save bandwidth and storage.

Back-of-Envelope Cost Analysis

Assuming 1 million users sending 100 messages/day with 20% containing media:

• Messages/day: 100 million
• Media uploads/day: 20 million
• Average media size: 1 MB -> 20 TB/day storage inflow
• Bandwidth: 20 million uploads + downloads -> ~5 Gbps sustained
• DB QPS: ~10,000 queries per second (writes + reads)
• Storage growth: 20 TB/day -> requires tiered storage and archival

Interview Tip

Start by defining scale milestones and expected traffic.

Identify the first bottleneck clearly (usually DB or storage).

Discuss incremental scaling steps: caching, read replicas, CDN, sharding.

Explain trade-offs and how each solution addresses specific bottlenecks.

Keep answers structured: current state, bottleneck, solution, future growth.

Self Check

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Add read replicas and implement caching to reduce direct DB load before scaling vertically or sharding.

Key Result

Media sharing systems first hit database bottlenecks at moderate scale, then storage and bandwidth limits at large scale; solutions include caching, read replicas, CDN, sharded storage, and horizontal scaling.

Practice

1. What is the main reason to separate media storage from message text storage in a messaging system?

easy

A. To reduce the number of messages sent

B. To make the user interface more colorful

C. To improve performance and scalability by handling large media files separately

D. To avoid using cloud storage services

Solution

1. Step 1: Understand media file size impact

   Media files like photos and videos are large and can slow down message storage if kept together.

2. Step 2: Separate storage benefits

   Separating media storage allows using specialized systems like CDNs and cloud storage optimized for large files.

3. Final Answer:

   To improve performance and scalability by handling large media files separately -> Option C

4. Quick Check:

   Separate media storage = better performance [OK]

Hint: Separate big files from text for faster message handling [OK]

Common Mistakes:

• Thinking media size doesn't affect performance
• Confusing UI design with storage design
• Ignoring scalability needs

2. Which component is essential for efficiently delivering media files in a messaging system?

easy

A. Content Delivery Network (CDN)

B. Message queue

C. Load balancer for text messages

D. Database index

Solution

1. Step 1: Identify media delivery needs

   Media files are large and need fast delivery to users worldwide.

2. Step 2: Role of CDN

   A CDN caches media files close to users, reducing latency and bandwidth usage.

3. Final Answer:

   Content Delivery Network (CDN) -> Option A

4. Quick Check:

   Fast media delivery = CDN [OK]

Hint: Use CDN to speed up media file delivery [OK]

Common Mistakes:

• Confusing CDN with database indexing
• Thinking load balancers handle media files
• Ignoring network latency

3. Given a messaging system where media files are uploaded to cloud storage and messages store only media URLs, what is the expected behavior when a user sends a photo?

medium

A. The photo is embedded directly in the message text stored in the database

B. The photo is uploaded to cloud storage and the message stores a URL pointing to it

C. The photo is sent as a base64 string inside the message

D. The photo is stored in the message queue before delivery

Solution

1. Step 1: Understand media upload flow

   Media files are large, so they are uploaded separately to cloud storage for efficiency.

2. Step 2: Message stores media reference

   The message contains a URL pointing to the media location, not the media itself.

3. Final Answer:

   The photo is uploaded to cloud storage and the message stores a URL pointing to it -> Option B

4. Quick Check:

   Media URL in message = cloud upload [OK]

Hint: Messages store URLs, not media files directly [OK]

Common Mistakes:

• Embedding large media in message text
• Using base64 which increases size
• Storing media in message queues

4. A messaging system stores media files directly in the message database causing slow queries. What is the best fix?

medium

A. Increase database RAM without changing design

B. Delete all media files from messages

C. Compress media files and keep them in the database

D. Move media files to cloud storage and store only URLs in the database

Solution

1. Step 1: Identify problem with current design

   Storing large media in the database slows down queries and reduces scalability.

2. Step 2: Apply best practice for media storage

   Moving media to cloud storage and storing URLs improves performance and scalability.

3. Final Answer:

   Move media files to cloud storage and store only URLs in the database -> Option D

4. Quick Check:

   Separate media storage = faster queries [OK]

Hint: Store URLs, not media, in message DB for speed [OK]

Common Mistakes:

• Relying only on hardware upgrades
• Compressing media but still storing in DB
• Removing media reduces feature, not fix

5. You design a messaging app that supports sharing photos and videos. To handle millions of users, which combination of components best ensures scalability and fast media delivery?

hard

A. Upload media to cloud storage, use a CDN for delivery, and store media URLs in messages

B. Store media in the message database and use a single server for all requests

C. Send media as base64 in messages and cache messages on client devices

D. Use peer-to-peer media sharing without central storage

Solution

1. Step 1: Analyze scalability needs

   Millions of users require scalable storage and fast delivery without overloading servers.

2. Step 2: Choose best architecture components

   Cloud storage handles large media files reliably; CDN speeds up delivery globally; storing URLs keeps messages lightweight.

3. Step 3: Evaluate other options

   Single server or embedding media causes bottlenecks; peer-to-peer lacks reliability and control.

4. Final Answer:

   Upload media to cloud storage, use a CDN for delivery, and store media URLs in messages -> Option A

5. Quick Check:

   Cloud storage + CDN + URLs = scalable media sharing [OK]

Hint: Combine cloud storage, CDN, and URLs for scalable media [OK]

Common Mistakes:

• Storing media in DB causing bottlenecks
• Sending base64 increases message size
• Ignoring global delivery speed