Netflix architecture overview in Microservices - Scalability & System Analysis

Scalability Analysis - Netflix architecture overview

Growth Table: Netflix Architecture Scaling

Scale	Users	Requests per Second	Data Volume	Infrastructure Changes
Small	100 users	~200 RPS	Few GBs of metadata	Single region, few microservices, basic caching
Medium	10,000 users	~20,000 RPS	TBs of metadata and video indexing	Multiple microservices, regional caching, CDN usage
Large	1,000,000 users	~2,000,000 RPS	Petabytes of video and metadata	Global CDN, multi-region deployment, microservice scaling, database sharding
Very Large	100,000,000 users	~200,000,000 RPS	Exabytes of data	Massive global distribution, advanced caching, multi-cloud, AI-driven load balancing

First Bottleneck

At small to medium scale, the database becomes the first bottleneck. Netflix stores user data, viewing history, and metadata which require fast reads and writes. As user count grows, the database faces high query loads and storage demands.

At large scale, network bandwidth and content delivery become bottlenecks due to massive video streaming traffic. The application servers and microservices also face CPU and memory pressure handling requests.

Scaling Solutions

• Database: Use sharding to split data across multiple databases. Employ read replicas to handle read-heavy workloads.
• Caching: Implement multi-layer caching (in-memory caches like Redis, CDN edge caches) to reduce database load and latency.
• Microservices: Scale horizontally by adding more instances behind load balancers. Use container orchestration (e.g., Kubernetes) for management.
• Content Delivery: Use a global CDN to serve video content close to users, reducing bandwidth and latency.
• Network: Optimize streaming protocols and compress data to reduce bandwidth usage.
• Multi-region Deployment: Deploy services in multiple geographic regions for fault tolerance and lower latency.

Back-of-Envelope Cost Analysis

For 1 million users streaming simultaneously:

• Requests per second: ~2 million (assuming 2 requests per second per user)
• Storage: Petabytes of video and metadata (Netflix stores thousands of movies and shows)
• Bandwidth: Video streaming at 3 Mbps per user -> 3 Mbps * 1M = 3 Tbps (~375 GB/s)
• Servers: Thousands of microservice instances and CDN edge servers globally

Interview Tip

Structure your scalability discussion by:

1. Identifying key components (database, microservices, CDN, network)
2. Estimating load and data growth at different scales
3. Pinpointing the first bottleneck and why it occurs
4. Proposing targeted scaling solutions for each bottleneck
5. Considering cost and complexity trade-offs
6. Discussing monitoring and fallback strategies

Self Check

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Add read replicas to distribute read queries and reduce load on the primary database. Also consider caching frequently accessed data to reduce database hits.

Key Result

Netflix architecture scales by evolving from a single-region microservice setup with a single database to a globally distributed system using sharded databases, multi-layer caching, and a global CDN to handle massive video streaming traffic efficiently.

Practice

1. What is the main reason Netflix uses microservices in its architecture?

easy

A. To make the system monolithic and simple

B. To use a single large database for all data

C. To avoid using APIs for communication

D. To break down the system into smaller, manageable parts

Solution

1. Step 1: Understand microservices purpose

   Microservices divide a large system into smaller parts that are easier to manage and update.

2. Step 2: Relate to Netflix architecture

   Netflix uses microservices to handle specific functions separately, improving scalability and maintenance.

3. Final Answer:

   To break down the system into smaller, manageable parts -> Option D

4. Quick Check:

   Microservices = Smaller parts [OK]

Hint: Microservices split big systems into small parts [OK]

Common Mistakes:

• Thinking microservices avoid APIs
• Believing Netflix uses one big database
• Confusing microservices with monolithic design

2. Which of the following correctly describes how Netflix microservices communicate?

easy

A. Through APIs that allow services to talk to each other

B. Using direct database connections between services

C. By sharing the same memory space

D. Using file system locks to coordinate

Solution

1. Step 1: Identify communication method in microservices

   Microservices communicate via APIs, which are defined interfaces for exchanging data.

2. Step 2: Match with Netflix architecture

   Netflix services use APIs to interact, ensuring loose coupling and independent deployment.

3. Final Answer:

   Through APIs that allow services to talk to each other -> Option A

4. Quick Check:

   Microservices communicate via APIs [OK]

Hint: Microservices talk via APIs, not direct DB or memory [OK]

Common Mistakes:

• Assuming services share memory
• Thinking services connect directly to databases
• Believing file locks coordinate services

3. Consider Netflix's microservice for user recommendations. If this service fails, what is the likely impact on the system?

medium

A. Only the recommendation feature will be affected

B. User login will fail for all users

C. The entire Netflix platform will stop working

D. Video streaming will be interrupted for all users

Solution

1. Step 1: Understand microservice isolation

   Each microservice handles a specific function independently, so failure affects only that function.

2. Step 2: Apply to recommendation service failure

   If the recommendation service fails, only recommendations stop working; other features like login or streaming continue.

3. Final Answer:

   Only the recommendation feature will be affected -> Option A

4. Quick Check:

   Microservice failure affects only its feature [OK]

Hint: Microservice failure affects only its own feature [OK]

Common Mistakes:

• Assuming entire platform fails
• Confusing recommendation with login or streaming
• Thinking microservices share failure impact

4. A developer notices that Netflix microservices are tightly coupled, causing deployment issues. What is the best fix?

medium

A. Increase the database size to handle more data

B. Refactor services to communicate only via APIs and avoid direct calls

C. Use shared global variables for communication

D. Merge all microservices into one big service

Solution

1. Step 1: Identify tight coupling problem

   Tightly coupled services depend directly on each other, causing deployment and scaling problems.

2. Step 2: Apply microservice best practice

   Services should communicate only via APIs to remain independent and deploy separately.

3. Final Answer:

   Refactor services to communicate only via APIs and avoid direct calls -> Option B

4. Quick Check:

   Loose coupling via APIs fixes deployment issues [OK]

Hint: Use APIs to keep services independent and deployable [OK]

Common Mistakes:

• Merging services defeats microservice benefits
• Using shared variables breaks isolation
• Increasing DB size doesn't fix coupling

5. Netflix wants to scale its video streaming microservice during peak hours. Which approach best fits its microservices architecture?

hard

A. Store all streaming data in a single database server

B. Combine streaming with user login service to reduce network calls

C. Deploy multiple instances of the streaming service behind a load balancer

D. Disable other microservices to free resources for streaming

Solution

1. Step 1: Understand scaling in microservices

   Scaling means running multiple copies of a service to handle more users.

2. Step 2: Apply to streaming service

   Deploying multiple streaming service instances with a load balancer distributes user requests efficiently.

3. Step 3: Evaluate other options

   Merging services or disabling others breaks microservice principles; single DB server is a bottleneck.

4. Final Answer:

   Deploy multiple instances of the streaming service behind a load balancer -> Option C

5. Quick Check:

   Scale by multiple instances + load balancer [OK]

Hint: Scale by adding instances and load balancer [OK]

Common Mistakes:

• Merging services reduces flexibility
• Single DB server limits scalability
• Disabling services harms user experience