Bird
Raised Fist0
HLDsystem_design~25 mins

Media sharing in messages in HLD - System Design Exercise

Choose your learning style10 modes available
LearnWhyDeepArchPracticeChallengeDesignRecallScale

Start learning this pattern below

Jump into concepts and practice - no test required

or
Recommended
Test this pattern10 questions across easy, medium, and hard to know if this pattern is strong
Design: Media Sharing in Messages
Design covers media upload, storage, message integration, delivery, and retrieval. Does not cover user authentication or message text-only features.
Functional Requirements
FR1: Users can send and receive messages with media attachments (images, videos, audio).
FR2: Support media upload, storage, and retrieval with messages.
FR3: Allow preview of media in message threads.
FR4: Support media formats: JPEG, PNG, MP4, MP3.
FR5: Ensure media is delivered with messages in real-time or near real-time.
FR6: Users can download or view media inline.
FR7: Support up to 10 million daily active users.
FR8: Handle up to 100,000 concurrent media uploads.
FR9: Ensure media upload latency p99 < 3 seconds.
FR10: Ensure message delivery latency p99 < 500ms.
FR11: Ensure 99.9% system availability.
Non-Functional Requirements
NFR1: Media files can be up to 50MB each.
NFR2: Storage must be scalable and cost-effective.
NFR3: System must protect user privacy and secure media content.
NFR4: Support mobile and web clients.
NFR5: Media should be cached for fast retrieval.
NFR6: Use CDN for media delivery to reduce latency.
Think Before You Design
Questions to Ask
❓ Question 1
❓ Question 2
❓ Question 3
❓ Question 4
❓ Question 5
❓ Question 6
❓ Question 7
Key Components
Client app for media upload and preview
API gateway or media upload service
Media processing service (thumbnail generation, transcoding)
Object storage (e.g., S3) for media files
CDN for media delivery
Message service to link media with messages
Cache layer for frequently accessed media metadata
Database for message and media metadata
Design Patterns
Asynchronous processing for media uploads
Content Delivery Network (CDN) for media caching
Microservices for separation of media and messaging
Event-driven architecture for media processing
Access control and signed URLs for secure media access
Reference Architecture
Client App
   |
   | Upload media + message
   v
API Gateway / Media Upload Service
   |
   | Store media file -> Object Storage (S3)
   | Send media info -> Media Processing Service
   | Store message + media metadata -> Message DB
   |
Media Processing Service
   |
   | Generate thumbnails, transcode if needed
   | Update media metadata in DB
   |
CDN <-> Object Storage
   |
Client App (media retrieval via CDN with signed URLs)

Cache Layer (Redis) for media metadata caching

Message Service handles message delivery with media references
Components
Client App
Mobile/Web
Upload media, send messages, preview and download media
API Gateway / Media Upload Service
REST/gRPC API
Receive media uploads, validate, and coordinate storage and processing
Media Processing Service
Microservice with worker queues
Generate thumbnails, transcode media, update metadata
Object Storage
Amazon S3 or equivalent
Store original media files and processed versions
Content Delivery Network (CDN)
Cloudflare, AWS CloudFront
Cache and deliver media files globally with low latency
Message Database
Relational DB (PostgreSQL)
Store messages and media metadata references
Cache Layer
Redis
Cache media metadata for fast access
Request Flow
1. 1. User selects media and message in client app and uploads to API Gateway.
2. 2. API Gateway validates media, stores file in Object Storage, and records metadata in Message DB.
3. 3. API Gateway sends media info to Media Processing Service asynchronously.
4. 4. Media Processing Service generates thumbnails and transcodes media if needed, updates metadata in DB.
5. 5. Message Service links media metadata with message and delivers message to recipients.
6. 6. Client retrieves media via CDN using signed URLs for secure access.
7. 7. Cache Layer stores frequently accessed media metadata to speed up retrieval.
Database Schema
Entities: - User (user_id, name, ...) - Message (message_id, sender_id, timestamp, text, ...) - Media (media_id, message_id, user_id, media_type, url, thumbnail_url, size, format, status, created_at) Relationships: - One Message can have zero or more Media attachments (1:N) - Media belongs to one User and one Message - Media metadata stored separately from message text for scalability
Scaling Discussion
Bottlenecks
High volume of concurrent media uploads can overwhelm upload service and storage.
Media processing can become a bottleneck if transcoding is slow.
Database can become a hotspot for media metadata reads/writes.
CDN cache misses can increase latency for media delivery.
Network bandwidth limits for large media downloads.
Solutions
Use load balancers and autoscaling for upload services to handle spikes.
Implement asynchronous processing with scalable worker pools for media processing.
Use database sharding or partitioning and caching to reduce DB load.
Leverage CDN with aggressive caching and cache invalidation strategies.
Use adaptive bitrate streaming for videos and compress media to reduce bandwidth.
Interview Tips
Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.
Clarify media types, sizes, and user scale early.
Separate media storage from message storage for scalability.
Use asynchronous processing to handle heavy media workloads.
Leverage CDN for fast media delivery and reduced latency.
Discuss security with signed URLs and access control.
Address bottlenecks with caching, autoscaling, and sharding.
Explain trade-offs between consistency and availability for media metadata.

Practice

(1/5)
1. What is the main reason to separate media storage from message text storage in a messaging system?
easy
A. To reduce the number of messages sent
B. To make the user interface more colorful
C. To improve performance and scalability by handling large media files separately
D. To avoid using cloud storage services

Solution

  1. Step 1: Understand media file size impact

    Media files like photos and videos are large and can slow down message storage if kept together.

  2. Step 2: Separate storage benefits

    Separating media storage allows using specialized systems like CDNs and cloud storage optimized for large files.

  3. Final Answer:

    To improve performance and scalability by handling large media files separately -> Option C

  4. Quick Check:

    Separate media storage = better performance [OK]
Hint: Separate big files from text for faster message handling [OK]
Common Mistakes:
  • Thinking media size doesn't affect performance
  • Confusing UI design with storage design
  • Ignoring scalability needs
2. Which component is essential for efficiently delivering media files in a messaging system?
easy
A. Content Delivery Network (CDN)
B. Message queue
C. Load balancer for text messages
D. Database index

Solution

  1. Step 1: Identify media delivery needs

    Media files are large and need fast delivery to users worldwide.

  2. Step 2: Role of CDN

    A CDN caches media files close to users, reducing latency and bandwidth usage.

  3. Final Answer:

    Content Delivery Network (CDN) -> Option A

  4. Quick Check:

    Fast media delivery = CDN [OK]
Hint: Use CDN to speed up media file delivery [OK]
Common Mistakes:
  • Confusing CDN with database indexing
  • Thinking load balancers handle media files
  • Ignoring network latency
3. Given a messaging system where media files are uploaded to cloud storage and messages store only media URLs, what is the expected behavior when a user sends a photo?
medium
A. The photo is embedded directly in the message text stored in the database
B. The photo is uploaded to cloud storage and the message stores a URL pointing to it
C. The photo is sent as a base64 string inside the message
D. The photo is stored in the message queue before delivery

Solution

  1. Step 1: Understand media upload flow

    Media files are large, so they are uploaded separately to cloud storage for efficiency.

  2. Step 2: Message stores media reference

    The message contains a URL pointing to the media location, not the media itself.

  3. Final Answer:

    The photo is uploaded to cloud storage and the message stores a URL pointing to it -> Option B

  4. Quick Check:

    Media URL in message = cloud upload [OK]
Hint: Messages store URLs, not media files directly [OK]
Common Mistakes:
  • Embedding large media in message text
  • Using base64 which increases size
  • Storing media in message queues
4. A messaging system stores media files directly in the message database causing slow queries. What is the best fix?
medium
A. Increase database RAM without changing design
B. Delete all media files from messages
C. Compress media files and keep them in the database
D. Move media files to cloud storage and store only URLs in the database

Solution

  1. Step 1: Identify problem with current design

    Storing large media in the database slows down queries and reduces scalability.

  2. Step 2: Apply best practice for media storage

    Moving media to cloud storage and storing URLs improves performance and scalability.

  3. Final Answer:

    Move media files to cloud storage and store only URLs in the database -> Option D

  4. Quick Check:

    Separate media storage = faster queries [OK]
Hint: Store URLs, not media, in message DB for speed [OK]
Common Mistakes:
  • Relying only on hardware upgrades
  • Compressing media but still storing in DB
  • Removing media reduces feature, not fix
5. You design a messaging app that supports sharing photos and videos. To handle millions of users, which combination of components best ensures scalability and fast media delivery?
hard
A. Upload media to cloud storage, use a CDN for delivery, and store media URLs in messages
B. Store media in the message database and use a single server for all requests
C. Send media as base64 in messages and cache messages on client devices
D. Use peer-to-peer media sharing without central storage

Solution

  1. Step 1: Analyze scalability needs

    Millions of users require scalable storage and fast delivery without overloading servers.

  2. Step 2: Choose best architecture components

    Cloud storage handles large media files reliably; CDN speeds up delivery globally; storing URLs keeps messages lightweight.

  3. Step 3: Evaluate other options

    Single server or embedding media causes bottlenecks; peer-to-peer lacks reliability and control.

  4. Final Answer:

    Upload media to cloud storage, use a CDN for delivery, and store media URLs in messages -> Option A

  5. Quick Check:

    Cloud storage + CDN + URLs = scalable media sharing [OK]
Hint: Combine cloud storage, CDN, and URLs for scalable media [OK]
Common Mistakes:
  • Storing media in DB causing bottlenecks
  • Sending base64 increases message size
  • Ignoring global delivery speed