Media sharing in messages in HLD - System Design Exercise

Design: Media Sharing in Messages

Design covers media upload, storage, message integration, delivery, and retrieval. Does not cover user authentication or message text-only features.

Functional Requirements

FR1: Users can send and receive messages with media attachments (images, videos, audio).

FR2: Support media upload, storage, and retrieval with messages.

FR3: Allow preview of media in message threads.

FR4: Support media formats: JPEG, PNG, MP4, MP3.

FR5: Ensure media is delivered with messages in real-time or near real-time.

FR6: Users can download or view media inline.

FR7: Support up to 10 million daily active users.

FR8: Handle up to 100,000 concurrent media uploads.

FR9: Ensure media upload latency p99 < 3 seconds.

FR10: Ensure message delivery latency p99 < 500ms.

FR11: Ensure 99.9% system availability.

Non-Functional Requirements

NFR1: Media files can be up to 50MB each.

NFR2: Storage must be scalable and cost-effective.

NFR3: System must protect user privacy and secure media content.

NFR4: Support mobile and web clients.

NFR5: Media should be cached for fast retrieval.

NFR6: Use CDN for media delivery to reduce latency.

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

❓ Question 7

Key Components

Client app for media upload and preview

API gateway or media upload service

Media processing service (thumbnail generation, transcoding)

Object storage (e.g., S3) for media files

CDN for media delivery

Message service to link media with messages

Cache layer for frequently accessed media metadata

Database for message and media metadata

Design Patterns

Asynchronous processing for media uploads

Content Delivery Network (CDN) for media caching

Microservices for separation of media and messaging

Event-driven architecture for media processing

Access control and signed URLs for secure media access

Reference Architecture

Client App
   |
   | Upload media + message
   v
API Gateway / Media Upload Service
   |
   | Store media file -> Object Storage (S3)
   | Send media info -> Media Processing Service
   | Store message + media metadata -> Message DB
   |
Media Processing Service
   |
   | Generate thumbnails, transcode if needed
   | Update media metadata in DB
   |
CDN <-> Object Storage
   |
Client App (media retrieval via CDN with signed URLs)

Cache Layer (Redis) for media metadata caching

Message Service handles message delivery with media references

Components

Client App

Mobile/Web

Upload media, send messages, preview and download media

API Gateway / Media Upload Service

REST/gRPC API

Receive media uploads, validate, and coordinate storage and processing

Media Processing Service

Microservice with worker queues

Generate thumbnails, transcode media, update metadata

Object Storage

Amazon S3 or equivalent

Store original media files and processed versions

Content Delivery Network (CDN)

Cloudflare, AWS CloudFront

Cache and deliver media files globally with low latency

Message Database

Relational DB (PostgreSQL)

Store messages and media metadata references

Cache Layer

Redis

Cache media metadata for fast access

Request Flow

1. 1. User selects media and message in client app and uploads to API Gateway.

2. 2. API Gateway validates media, stores file in Object Storage, and records metadata in Message DB.

3. 3. API Gateway sends media info to Media Processing Service asynchronously.

4. 4. Media Processing Service generates thumbnails and transcodes media if needed, updates metadata in DB.

5. 5. Message Service links media metadata with message and delivers message to recipients.

6. 6. Client retrieves media via CDN using signed URLs for secure access.

7. 7. Cache Layer stores frequently accessed media metadata to speed up retrieval.

Database Schema

Entities: - User (user_id, name, ...) - Message (message_id, sender_id, timestamp, text, ...) - Media (media_id, message_id, user_id, media_type, url, thumbnail_url, size, format, status, created_at) Relationships: - One Message can have zero or more Media attachments (1:N) - Media belongs to one User and one Message - Media metadata stored separately from message text for scalability

Scaling Discussion

Bottlenecks

High volume of concurrent media uploads can overwhelm upload service and storage.

Media processing can become a bottleneck if transcoding is slow.

Database can become a hotspot for media metadata reads/writes.

CDN cache misses can increase latency for media delivery.

Network bandwidth limits for large media downloads.

Solutions

Use load balancers and autoscaling for upload services to handle spikes.

Implement asynchronous processing with scalable worker pools for media processing.

Use database sharding or partitioning and caching to reduce DB load.

Leverage CDN with aggressive caching and cache invalidation strategies.

Use adaptive bitrate streaming for videos and compress media to reduce bandwidth.

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Clarify media types, sizes, and user scale early.

Separate media storage from message storage for scalability.

Use asynchronous processing to handle heavy media workloads.

Leverage CDN for fast media delivery and reduced latency.

Discuss security with signed URLs and access control.

Address bottlenecks with caching, autoscaling, and sharding.

Explain trade-offs between consistency and availability for media metadata.

Practice

1. What is the main reason to separate media storage from message text storage in a messaging system?

easy

A. To reduce the number of messages sent

B. To make the user interface more colorful

C. To improve performance and scalability by handling large media files separately

D. To avoid using cloud storage services

Solution

1. Step 1: Understand media file size impact

   Media files like photos and videos are large and can slow down message storage if kept together.

2. Step 2: Separate storage benefits

   Separating media storage allows using specialized systems like CDNs and cloud storage optimized for large files.

3. Final Answer:

   To improve performance and scalability by handling large media files separately -> Option C

4. Quick Check:

   Separate media storage = better performance [OK]

Hint: Separate big files from text for faster message handling [OK]

Common Mistakes:

• Thinking media size doesn't affect performance
• Confusing UI design with storage design
• Ignoring scalability needs

2. Which component is essential for efficiently delivering media files in a messaging system?

easy

A. Content Delivery Network (CDN)

B. Message queue

C. Load balancer for text messages

D. Database index

Solution

1. Step 1: Identify media delivery needs

   Media files are large and need fast delivery to users worldwide.

2. Step 2: Role of CDN

   A CDN caches media files close to users, reducing latency and bandwidth usage.

3. Final Answer:

   Content Delivery Network (CDN) -> Option A

4. Quick Check:

   Fast media delivery = CDN [OK]

Hint: Use CDN to speed up media file delivery [OK]

Common Mistakes:

• Confusing CDN with database indexing
• Thinking load balancers handle media files
• Ignoring network latency

3. Given a messaging system where media files are uploaded to cloud storage and messages store only media URLs, what is the expected behavior when a user sends a photo?

medium

A. The photo is embedded directly in the message text stored in the database

B. The photo is uploaded to cloud storage and the message stores a URL pointing to it

C. The photo is sent as a base64 string inside the message

D. The photo is stored in the message queue before delivery

Solution

1. Step 1: Understand media upload flow

   Media files are large, so they are uploaded separately to cloud storage for efficiency.

2. Step 2: Message stores media reference

   The message contains a URL pointing to the media location, not the media itself.

3. Final Answer:

   The photo is uploaded to cloud storage and the message stores a URL pointing to it -> Option B

4. Quick Check:

   Media URL in message = cloud upload [OK]

Hint: Messages store URLs, not media files directly [OK]

Common Mistakes:

• Embedding large media in message text
• Using base64 which increases size
• Storing media in message queues

4. A messaging system stores media files directly in the message database causing slow queries. What is the best fix?

medium

A. Increase database RAM without changing design

B. Delete all media files from messages

C. Compress media files and keep them in the database

D. Move media files to cloud storage and store only URLs in the database

Solution

1. Step 1: Identify problem with current design

   Storing large media in the database slows down queries and reduces scalability.

2. Step 2: Apply best practice for media storage

   Moving media to cloud storage and storing URLs improves performance and scalability.

3. Final Answer:

   Move media files to cloud storage and store only URLs in the database -> Option D

4. Quick Check:

   Separate media storage = faster queries [OK]

Hint: Store URLs, not media, in message DB for speed [OK]

Common Mistakes:

• Relying only on hardware upgrades
• Compressing media but still storing in DB
• Removing media reduces feature, not fix

5. You design a messaging app that supports sharing photos and videos. To handle millions of users, which combination of components best ensures scalability and fast media delivery?

hard

A. Upload media to cloud storage, use a CDN for delivery, and store media URLs in messages

B. Store media in the message database and use a single server for all requests

C. Send media as base64 in messages and cache messages on client devices

D. Use peer-to-peer media sharing without central storage

Solution

1. Step 1: Analyze scalability needs

   Millions of users require scalable storage and fast delivery without overloading servers.

2. Step 2: Choose best architecture components

   Cloud storage handles large media files reliably; CDN speeds up delivery globally; storing URLs keeps messages lightweight.

3. Step 3: Evaluate other options

   Single server or embedding media causes bottlenecks; peer-to-peer lacks reliability and control.

4. Final Answer:

   Upload media to cloud storage, use a CDN for delivery, and store media URLs in messages -> Option A

5. Quick Check:

   Cloud storage + CDN + URLs = scalable media sharing [OK]

Hint: Combine cloud storage, CDN, and URLs for scalable media [OK]

Common Mistakes:

• Storing media in DB causing bottlenecks
• Sending base64 increases message size
• Ignoring global delivery speed