Media sharing in messages in HLD - Deep Dive

Overview - Media sharing in messages

What is it?

Media sharing in messages means sending pictures, videos, audio, or files along with text in a chat or messaging app. It allows users to express themselves better and share moments instantly. This feature handles storing, transferring, and displaying media smoothly between users. It works behind the scenes to keep media safe, fast, and easy to access.

Why it matters

Without media sharing, messaging would be plain and less engaging, like talking without showing pictures or videos. People rely on media to communicate emotions, share memories, or collaborate. Media sharing solves the problem of sending large files reliably and quickly over networks. It also ensures media is stored securely and loads well on different devices, making conversations richer and more meaningful.

Where it fits

Before learning media sharing, you should understand basic messaging systems and data transfer concepts. After this, you can explore advanced topics like content delivery networks, media compression, and real-time streaming. This topic fits into the broader journey of building scalable, user-friendly communication platforms.

Mental Model

Core Idea

Media sharing in messages is like sending a letter with photos and gifts inside, where the system safely packs, sends, and delivers these extras along with the words.

Think of it like...

Imagine mailing a postcard with a photo and a small gift inside an envelope. The postal service must handle the extra weight and size carefully, ensure it reaches the right person, and keep it safe until delivery.

┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│ User Device   │─────▶│ Media Storage │─────▶│ Recipient     │
│ (Compose Msg) │      │ (Store Files) │      │ Device       │
└───────────────┘      └───────────────┘      └───────────────┘
       │                      ▲                      │
       │                      │                      │
       ▼                      │                      ▼
┌───────────────┐             │             ┌───────────────┐
│ Media Upload  │─────────────┘             │ Media Download│
│ & Encoding   │                           │ & Display     │
└───────────────┘                           └───────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding basic message content

Concept: Messages can contain text and media, which need different handling.

Text messages are simple strings sent over the network. Media like images or videos are larger and binary, requiring special storage and transfer methods. The system must recognize media types and handle them separately from text.

Result

Learners understand that media is not just text and needs different processing.

Knowing that media differs from text is key to designing systems that handle both efficiently.

2

Foundation

Media file types and sizes

Concept: Media comes in various formats and sizes, affecting storage and transfer.

Images (JPEG, PNG), videos (MP4, WebM), audio (MP3, AAC), and documents have different sizes and compression. Large files take more bandwidth and storage. Systems must support multiple formats and optimize for size and quality.

Result

Learners grasp the diversity and challenges of handling media files.

Understanding media variety helps in planning storage and bandwidth needs.

3

Intermediate

Uploading and storing media securely

🤔Before reading on: do you think media files are stored directly inside messages or separately? Commit to your answer.

Concept: Media files are uploaded separately and stored securely, with references in messages.

When a user sends media, the file uploads to a dedicated storage service. The message contains a link or ID pointing to the stored media. This separation improves performance and security. Storage uses encryption and access controls to protect media.

Result

Learners see how media is managed separately from text for efficiency and safety.

Knowing media is stored separately prevents common design mistakes that slow down messaging.

4

Intermediate

Media delivery and caching strategies

🤔Before reading on: do you think every media request fetches the file from storage or uses cached copies? Commit to your answer.

Concept: Caching media closer to users speeds up delivery and reduces load on storage.

Systems use content delivery networks (CDNs) to cache media files globally. When a recipient requests media, it loads from the nearest cache, reducing delay. This approach handles high traffic and improves user experience.

Result

Learners understand how caching optimizes media delivery.

Recognizing caching's role helps design scalable and fast media sharing.

5

Intermediate

Handling media format conversion and thumbnails

Concept: Media is often converted and previewed to save bandwidth and improve UI.

Large videos or images are converted to smaller sizes or different formats for preview. Thumbnails are generated for quick display in message lists. This reduces data usage and speeds up loading on devices.

Result

Learners see how media is optimized for different devices and network conditions.

Understanding media processing improves user experience and resource use.

6

Advanced

Ensuring media security and privacy

🤔Before reading on: do you think media files are public by default or access-controlled? Commit to your answer.

Concept: Media must be protected with encryption and access controls to maintain privacy.

Media storage uses encryption at rest and in transit. Access tokens or signed URLs ensure only authorized users can view media. Systems audit access and prevent unauthorized sharing or leaks.

Result

Learners appreciate the importance of security in media sharing.

Knowing security measures prevents data breaches and builds user trust.

7

Expert

Scaling media sharing for millions of users

🤔Before reading on: do you think a single server can handle all media uploads and downloads for a large app? Commit to your answer.

Concept: Large-scale media sharing requires distributed storage, load balancing, and fault tolerance.

Systems use distributed object storage (like S3), multiple CDN providers, and load balancers to handle massive traffic. They implement replication and failover to ensure availability. Monitoring and auto-scaling keep performance stable under load.

Result

Learners understand the complexity behind reliable, large-scale media sharing.

Understanding scaling challenges prepares learners for real-world system design.

Under the Hood

When a user sends media, the client uploads the file to a storage service using secure protocols. The storage service saves the file with metadata and generates access URLs. The messaging system stores a reference to this media in the message data. When recipients open the message, their clients fetch the media from storage or CDN using the URL. Media files are often encrypted and access-controlled to protect privacy. Caching layers reduce latency by serving media from locations near users. The system monitors usage and scales storage and delivery infrastructure dynamically.

Why designed this way?

Separating media storage from message data avoids bloating message databases and improves performance. Using CDNs and caching reduces latency and bandwidth costs. Encryption and access control protect user privacy and comply with regulations. Distributed storage and load balancing ensure the system can handle millions of users without downtime. These design choices balance speed, security, scalability, and cost.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ User Client   │──────▶│ Upload Server │──────▶│ Object Storage│
│ (Send Media)  │       │ (Handles Auth)│       │ (Stores Files)│
└───────────────┘       └───────────────┘       └───────────────┘
       │                        ▲                       │
       │                        │                       ▼
       │                ┌───────────────┐       ┌───────────────┐
       │                │ CDN / Cache   │◀──────│ Media Request │
       ▼                └───────────────┘       │ from Recipient│
┌───────────────┐                               └───────────────┘
│ Messaging DB  │
│ (Stores Msgs) │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Is media stored directly inside the message text? Commit to yes or no.

Common Belief: Media files are stored inside the message data along with text.

Reality: Media is stored separately in dedicated storage, and messages contain only references or links.

Why it matters: Storing media inside messages would slow down message retrieval and increase database size, causing poor performance.

Quick: Do you think caching media is optional or essential for performance? Commit to your answer.

Common Belief: Caching media is optional and does not affect user experience much.

Reality: Caching media near users is essential to reduce latency and bandwidth usage, especially for large files.

Why it matters: Without caching, media loads slowly and increases server load, degrading user experience.

Quick: Are media files public by default or protected? Commit to your answer.

Common Belief: Media files are public and accessible by anyone with the link.

Reality: Media files are protected with access controls and encryption to ensure privacy.

Why it matters: Public media can lead to privacy breaches and unauthorized sharing.

Quick: Can a single server handle all media uploads for a large app? Commit to yes or no.

Common Belief: One server can handle all media uploads and downloads for any app size.

Reality: Large apps require distributed storage and load balancing to handle massive media traffic.

Why it matters: Relying on one server causes bottlenecks, downtime, and poor scalability.

Expert Zone

1

Media URLs often use short-lived signed tokens to prevent unauthorized sharing, a detail many overlook.

2

Thumbnail generation must balance quality and size carefully to optimize user experience without wasting resources.

3

Handling media metadata (like orientation, duration) correctly is crucial for proper display but often ignored.

When NOT to use

For very small or text-only messaging apps, complex media storage and CDN setups may be overkill. Instead, simple base64 encoding or inline media might suffice. Also, for real-time streaming media, specialized streaming protocols and servers are better than static file storage.

Production Patterns

Popular messaging apps use cloud object storage (e.g., AWS S3), CDNs (e.g., Cloudflare), and microservices for media processing. They implement encryption, signed URLs, and multi-format transcoding. Auto-scaling and monitoring ensure smooth media sharing even during traffic spikes.

Connections

Content Delivery Networks (CDNs)

Media sharing builds on CDN technology to deliver files quickly.

Understanding CDNs clarifies how media reaches users fast and reliably.

Encryption and Access Control

Media sharing depends on encryption and access control to protect privacy.

Knowing security principles helps design safe media sharing systems.

Postal Mail System

Both systems handle sending physical or digital items safely and efficiently.

Comparing media sharing to postal mail reveals the importance of packaging, routing, and delivery guarantees.

Common Pitfalls

#1 Embedding media files directly inside message text data.

Wrong approach:

Store entire image or video binary inside the message database field.

Correct approach:

Upload media to object storage and store only a reference URL in the message.

Root cause:

Misunderstanding the difference between message metadata and large binary data handling.

#2 Not using caching or CDNs for media delivery.

Wrong approach:

Serve media files directly from the storage server for every request.

Correct approach:

Use a CDN to cache media files close to users for faster delivery.

Root cause:

Underestimating the impact of latency and bandwidth on user experience.

#3 Making media URLs public without access control.

Wrong approach:

Generate static URLs accessible by anyone without authentication.

Correct approach:

Use signed URLs or tokens that expire and require authorization.

Root cause:

Ignoring security and privacy requirements for user data.

Key Takeaways

Media sharing enriches messaging by allowing users to send images, videos, and files alongside text.

Media files are stored separately from messages to optimize performance and scalability.

Caching and CDNs are critical to delivering media quickly and reliably to users worldwide.

Security measures like encryption and access control protect user privacy in media sharing.

Scaling media sharing requires distributed storage, load balancing, and careful resource management.

Practice

1. What is the main reason to separate media storage from message text storage in a messaging system?

easy

A. To reduce the number of messages sent

B. To make the user interface more colorful

C. To improve performance and scalability by handling large media files separately

D. To avoid using cloud storage services

Solution

1. Step 1: Understand media file size impact

   Media files like photos and videos are large and can slow down message storage if kept together.

2. Step 2: Separate storage benefits

   Separating media storage allows using specialized systems like CDNs and cloud storage optimized for large files.

3. Final Answer:

   To improve performance and scalability by handling large media files separately -> Option C

4. Quick Check:

   Separate media storage = better performance [OK]

Hint: Separate big files from text for faster message handling [OK]

Common Mistakes:

• Thinking media size doesn't affect performance
• Confusing UI design with storage design
• Ignoring scalability needs

2. Which component is essential for efficiently delivering media files in a messaging system?

easy

A. Content Delivery Network (CDN)

B. Message queue

C. Load balancer for text messages

D. Database index

Solution

1. Step 1: Identify media delivery needs

   Media files are large and need fast delivery to users worldwide.

2. Step 2: Role of CDN

   A CDN caches media files close to users, reducing latency and bandwidth usage.

3. Final Answer:

   Content Delivery Network (CDN) -> Option A

4. Quick Check:

   Fast media delivery = CDN [OK]

Hint: Use CDN to speed up media file delivery [OK]

Common Mistakes:

• Confusing CDN with database indexing
• Thinking load balancers handle media files
• Ignoring network latency

3. Given a messaging system where media files are uploaded to cloud storage and messages store only media URLs, what is the expected behavior when a user sends a photo?

medium

A. The photo is embedded directly in the message text stored in the database

B. The photo is uploaded to cloud storage and the message stores a URL pointing to it

C. The photo is sent as a base64 string inside the message

D. The photo is stored in the message queue before delivery

Solution

1. Step 1: Understand media upload flow

   Media files are large, so they are uploaded separately to cloud storage for efficiency.

2. Step 2: Message stores media reference

   The message contains a URL pointing to the media location, not the media itself.

3. Final Answer:

   The photo is uploaded to cloud storage and the message stores a URL pointing to it -> Option B

4. Quick Check:

   Media URL in message = cloud upload [OK]

Hint: Messages store URLs, not media files directly [OK]

Common Mistakes:

• Embedding large media in message text
• Using base64 which increases size
• Storing media in message queues

4. A messaging system stores media files directly in the message database causing slow queries. What is the best fix?

medium

A. Increase database RAM without changing design

B. Delete all media files from messages

C. Compress media files and keep them in the database

D. Move media files to cloud storage and store only URLs in the database

Solution

1. Step 1: Identify problem with current design

   Storing large media in the database slows down queries and reduces scalability.

2. Step 2: Apply best practice for media storage

   Moving media to cloud storage and storing URLs improves performance and scalability.

3. Final Answer:

   Move media files to cloud storage and store only URLs in the database -> Option D

4. Quick Check:

   Separate media storage = faster queries [OK]

Hint: Store URLs, not media, in message DB for speed [OK]

Common Mistakes:

• Relying only on hardware upgrades
• Compressing media but still storing in DB
• Removing media reduces feature, not fix

5. You design a messaging app that supports sharing photos and videos. To handle millions of users, which combination of components best ensures scalability and fast media delivery?

hard

A. Upload media to cloud storage, use a CDN for delivery, and store media URLs in messages

B. Store media in the message database and use a single server for all requests

C. Send media as base64 in messages and cache messages on client devices

D. Use peer-to-peer media sharing without central storage

Solution

1. Step 1: Analyze scalability needs

   Millions of users require scalable storage and fast delivery without overloading servers.

2. Step 2: Choose best architecture components

   Cloud storage handles large media files reliably; CDN speeds up delivery globally; storing URLs keeps messages lightweight.

3. Step 3: Evaluate other options

   Single server or embedding media causes bottlenecks; peer-to-peer lacks reliability and control.

4. Final Answer:

   Upload media to cloud storage, use a CDN for delivery, and store media URLs in messages -> Option A

5. Quick Check:

   Cloud storage + CDN + URLs = scalable media sharing [OK]

Hint: Combine cloud storage, CDN, and URLs for scalable media [OK]

Common Mistakes:

• Storing media in DB causing bottlenecks
• Sending base64 increases message size
• Ignoring global delivery speed