Why scaling requires different strategies in MLOps - See It in Action

Why Scaling Requires Different Strategies

📖 Scenario: You are working in a team that manages machine learning models in production. As the number of users grows, you notice that the current setup struggles to handle the load. You need to understand why scaling your system requires different strategies depending on the situation.

🎯 Goal: Build a simple Python program that models different scaling strategies and shows how they affect system capacity. This will help you understand why one size does not fit all when scaling machine learning systems.

📋 What You'll Learn

Create a dictionary called systems with three keys: 'single_node', 'horizontal_scaling', and 'vertical_scaling' with values 1, 5, and 2 respectively.

Create a variable called load and set it to 7.

Use a for loop with variables strategy and capacity to iterate over systems.items() and create a new dictionary can_handle_load where each key is the strategy and the value is true if capacity is greater than or equal to load, otherwise false.

Print the can_handle_load dictionary to show which strategies can handle the load.

💡 Why This Matters

🌍 Real World

In real machine learning operations, systems must scale to handle more users or data. Different scaling strategies like adding more machines (horizontal) or upgrading a machine (vertical) have different limits and costs.

💼 Career

Understanding scaling strategies is key for MLOps engineers to keep machine learning models running smoothly as demand grows.

1

Create the initial system capacities

Create a dictionary called systems with these exact entries: 'single_node': 1, 'horizontal_scaling': 5, and 'vertical_scaling': 2.

MLOps

# Create the systems dictionary with exact capacities
# Your code here

Hint

Use curly braces to create a dictionary with the exact keys and values.

2

Set the current load value

Create a variable called load and set it to 7.

MLOps

systems = {'single_node': 1, 'horizontal_scaling': 5, 'vertical_scaling': 2}
# Set the load variable to 7
# Your code here

Hint

Just assign the number 7 to the variable named load.

3

Determine which strategies can handle the load

Use a for loop with variables strategy and capacity to iterate over systems.items(). Inside the loop, create a dictionary called can_handle_load where each key is the strategy and the value is true if capacity is greater than or equal to load, otherwise false.

MLOps

systems = {'single_node': 1, 'horizontal_scaling': 5, 'vertical_scaling': 2}
load = 7
# Create can_handle_load dictionary using a for loop
# Your code here

Hint

Start with an empty dictionary can_handle_load = {}. Then use a for loop to fill it.

4

Print the results

Write print(can_handle_load) to display which scaling strategies can handle the load.

MLOps

systems = {'single_node': 1, 'horizontal_scaling': 5, 'vertical_scaling': 2}
load = 7
can_handle_load = {}
for strategy, capacity in systems.items():
    can_handle_load[strategy] = capacity >= load
# Print the can_handle_load dictionary
# Your code here

Hint

Use the print function to show the dictionary.

Practice

1. Why do systems need different scaling strategies as they grow?

easy

A. Because all systems grow at the same speed

B. Because scaling always means adding more machines

C. Because different growth patterns require different resource management

D. Because vertical scaling is always better than horizontal scaling

Solution

1. Step 1: Understand system growth patterns

   Systems grow in different ways, such as more users or more data, which affects resource needs differently.

2. Step 2: Match scaling strategy to growth type

   Different growth types require different scaling approaches to manage resources efficiently and keep performance.

3. Final Answer:

   Because different growth patterns require different resource management -> Option C

4. Quick Check:

   Growth patterns = Different strategies [OK]

Hint: Match scaling to how system grows for best results [OK]

Common Mistakes:

• Assuming one scaling method fits all
• Thinking scaling always means adding machines
• Ignoring resource limits of single machines

2. Which of the following is the correct way to describe vertical scaling?

easy

A. Adding more machines to handle more load

B. Making a single machine more powerful by adding CPU or RAM

C. Splitting data across multiple databases

D. Reducing the number of users on the system

Solution

1. Step 1: Define vertical scaling

   Vertical scaling means improving one machine's capacity by adding resources like CPU or memory.

2. Step 2: Compare options

   Making a single machine more powerful by adding CPU or RAM matches this definition; others describe horizontal scaling or unrelated actions.

3. Final Answer:

   Making a single machine more powerful by adding CPU or RAM -> Option B

4. Quick Check:

   Vertical scaling = stronger single machine [OK]

Hint: Vertical scaling = upgrade one machine's power [OK]

Common Mistakes:

• Confusing vertical with horizontal scaling
• Thinking vertical scaling means adding machines
• Selecting unrelated options like reducing users

3. Consider a system that uses horizontal scaling by adding identical servers behind a load balancer. What is the main benefit of this approach?

medium

A. It allows the system to handle more users by distributing load

B. It simplifies the software by using only one server

C. It reduces the need for network connections

D. It increases the power of a single server

Solution

1. Step 1: Understand horizontal scaling

   Horizontal scaling adds more servers to share the workload, improving capacity.

2. Step 2: Identify benefit of load balancing

   Load balancers distribute user requests across servers, allowing more users to be served efficiently.

3. Final Answer:

   It allows the system to handle more users by distributing load -> Option A

4. Quick Check:

   Horizontal scaling = distribute load [OK]

Hint: More servers = more users handled [OK]

Common Mistakes:

• Thinking horizontal scaling powers one server
• Believing it reduces network needs
• Assuming it simplifies software to one server

4. A team tried to scale their ML model serving by only upgrading the CPU and RAM of one server, but the system still slowed down under heavy user load. What is the likely problem?

medium

A. They must have a bug in the model code

B. They needed to reduce the model size instead

C. They should have used a faster programming language

D. They should have added more servers instead of upgrading one

Solution

1. Step 1: Analyze the scaling approach

   Upgrading one server is vertical scaling, which has limits and may not handle very high loads.

2. Step 2: Identify better scaling strategy

   Adding more servers (horizontal scaling) distributes load and improves performance under heavy use.

3. Final Answer:

   They should have added more servers instead of upgrading one -> Option D

4. Quick Check:

   Heavy load needs horizontal scaling [OK]

Hint: Heavy load? Add servers, not just power [OK]

Common Mistakes:

• Blaming model size without checking scaling
• Assuming programming language causes slowdown
• Ignoring scaling limits of single server

5. You manage an ML system that processes large datasets and serves predictions to many users. Vertical scaling is costly and limited. Which combined strategy best balances cost, performance, and reliability?

hard

A. Use horizontal scaling with multiple servers and optimize model efficiency

B. Only upgrade the biggest server continuously

C. Reduce the number of users to fit one server

D. Switch to a simpler model without scaling

Solution

1. Step 1: Evaluate vertical scaling limits

   Vertical scaling is costly and hits hardware limits, so relying on it alone is not sustainable.

2. Step 2: Combine horizontal scaling and optimization

   Adding servers (horizontal scaling) spreads load, while optimizing the model reduces resource use, balancing cost and performance.

3. Step 3: Consider reliability

   Multiple servers improve fault tolerance, making the system more reliable than a single powerful server.

4. Final Answer:

   Use horizontal scaling with multiple servers and optimize model efficiency -> Option A

5. Quick Check:

   Combine horizontal scaling + optimization = best balance [OK]

Hint: Combine adding servers with model optimization [OK]

Common Mistakes:

• Relying only on vertical scaling
• Ignoring user demand growth
• Choosing to reduce users instead of scaling
• Dropping scaling for simpler models only