Media sharing in messages in HLD - System Design Exercise

Design: Media Sharing in Messages

Design covers media upload, storage, message integration, delivery, and retrieval. Does not cover user authentication or message text-only features.

Functional Requirements

FR1: Users can send and receive messages with media attachments (images, videos, audio).

FR2: Support media upload, storage, and retrieval with messages.

FR3: Allow preview of media in message threads.

FR4: Support media formats: JPEG, PNG, MP4, MP3.

FR5: Ensure media is delivered with messages in real-time or near real-time.

FR6: Users can download or view media inline.

FR7: Support up to 10 million daily active users.

FR8: Handle up to 100,000 concurrent media uploads.

FR9: Ensure media upload latency p99 < 3 seconds.

FR10: Ensure message delivery latency p99 < 500ms.

FR11: Ensure 99.9% system availability.

Non-Functional Requirements

NFR1: Media files can be up to 50MB each.

NFR2: Storage must be scalable and cost-effective.

NFR3: System must protect user privacy and secure media content.

NFR4: Support mobile and web clients.

NFR5: Media should be cached for fast retrieval.

NFR6: Use CDN for media delivery to reduce latency.

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

❓ Question 7

Key Components

Client app for media upload and preview

API gateway or media upload service

Media processing service (thumbnail generation, transcoding)

Object storage (e.g., S3) for media files

CDN for media delivery

Message service to link media with messages

Cache layer for frequently accessed media metadata

Database for message and media metadata

Design Patterns

Asynchronous processing for media uploads

Content Delivery Network (CDN) for media caching

Microservices for separation of media and messaging

Event-driven architecture for media processing

Access control and signed URLs for secure media access

Reference Architecture

Client App
   |
   | Upload media + message
   v
API Gateway / Media Upload Service
   |
   | Store media file -> Object Storage (S3)
   | Send media info -> Media Processing Service
   | Store message + media metadata -> Message DB
   |
Media Processing Service
   |
   | Generate thumbnails, transcode if needed
   | Update media metadata in DB
   |
CDN <-> Object Storage
   |
Client App (media retrieval via CDN with signed URLs)

Cache Layer (Redis) for media metadata caching

Message Service handles message delivery with media references

Components

Client App

Mobile/Web

Upload media, send messages, preview and download media

API Gateway / Media Upload Service

REST/gRPC API

Receive media uploads, validate, and coordinate storage and processing

Media Processing Service

Microservice with worker queues

Generate thumbnails, transcode media, update metadata

Object Storage

Amazon S3 or equivalent

Store original media files and processed versions

Content Delivery Network (CDN)

Cloudflare, AWS CloudFront

Cache and deliver media files globally with low latency

Message Database

Relational DB (PostgreSQL)

Store messages and media metadata references

Cache Layer

Redis

Cache media metadata for fast access

Request Flow

1. 1. User selects media and message in client app and uploads to API Gateway.

2. 2. API Gateway validates media, stores file in Object Storage, and records metadata in Message DB.

3. 3. API Gateway sends media info to Media Processing Service asynchronously.

4. 4. Media Processing Service generates thumbnails and transcodes media if needed, updates metadata in DB.

5. 5. Message Service links media metadata with message and delivers message to recipients.

6. 6. Client retrieves media via CDN using signed URLs for secure access.

7. 7. Cache Layer stores frequently accessed media metadata to speed up retrieval.

Database Schema

Entities: - User (user_id, name, ...) - Message (message_id, sender_id, timestamp, text, ...) - Media (media_id, message_id, user_id, media_type, url, thumbnail_url, size, format, status, created_at) Relationships: - One Message can have zero or more Media attachments (1:N) - Media belongs to one User and one Message - Media metadata stored separately from message text for scalability

Scaling Discussion

Bottlenecks

High volume of concurrent media uploads can overwhelm upload service and storage.

Media processing can become a bottleneck if transcoding is slow.

Database can become a hotspot for media metadata reads/writes.

CDN cache misses can increase latency for media delivery.

Network bandwidth limits for large media downloads.

Solutions

Use load balancers and autoscaling for upload services to handle spikes.

Implement asynchronous processing with scalable worker pools for media processing.

Use database sharding or partitioning and caching to reduce DB load.

Leverage CDN with aggressive caching and cache invalidation strategies.

Use adaptive bitrate streaming for videos and compress media to reduce bandwidth.

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Clarify media types, sizes, and user scale early.

Separate media storage from message storage for scalability.

Use asynchronous processing to handle heavy media workloads.

Leverage CDN for fast media delivery and reduced latency.

Discuss security with signed URLs and access control.

Address bottlenecks with caching, autoscaling, and sharding.

Explain trade-offs between consistency and availability for media metadata.