Why scaling requires different strategies in MLOps - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why horizontal scaling differs from vertical scaling

Which statement best explains why horizontal scaling requires different strategies than vertical scaling in machine learning operations?

A Horizontal scaling adds more machines, so it needs coordination between them, while vertical scaling upgrades a single machine's resources.

B Vertical scaling is cheaper and faster than horizontal scaling, so it always uses the same strategy.

C Horizontal scaling only works with cloud services, while vertical scaling is for on-premises only.

D Vertical scaling duplicates data across machines, while horizontal scaling increases CPU speed.

💡 Hint

Think about how adding more machines versus upgrading one machine affects system design.

Horizontal scaling means adding more machines that must work together, requiring strategies for coordination and data sharing. Vertical scaling means making one machine stronger, which doesn't need coordination.

💻 Command Output

intermediate

Output of Kubernetes pod scaling command

What is the output of this command when scaling a deployment named 'ml-model' to 3 replicas?

MLOps

kubectl scale deployment ml-model --replicas=3

A deployment.apps/ml-model scaled

B Error from server (NotFound): deployments.apps "ml-model" not found

C deployment.apps/ml-model created

D No resources found in default namespace.

💡 Hint

Scaling a deployment changes the number of pods, not creating or deleting the deployment.

The command scales the existing deployment to the specified number of replicas and confirms with 'deployment.apps/ml-model scaled'.

🔀 Workflow

advanced

Order the steps to implement auto-scaling for a machine learning service

Put these steps in the correct order to set up auto-scaling for a machine learning model deployment.

A 3,2,1,4

B 3,1,2,4

C 1,3,2,4

D 1,2,3,4

💡 Hint

Think about deploying first, then monitoring and configuring scaling.

You first deploy the service, then define what to monitor, configure scaling rules, and finally test the setup.

❓ Troubleshoot

advanced

Troubleshooting failed scaling in Kubernetes

You tried to scale a deployment but the pods remain at the old count. Which issue is most likely causing this?

A The deployment YAML file is missing the container image.

B The pods are stuck in CrashLoopBackOff due to code errors.

C The Kubernetes cluster is offline.

D The deployment has a resource quota limiting pod count.

💡 Hint

Think about cluster limits that prevent scaling.

A resource quota can limit how many pods can run, preventing scaling even if the command succeeds.

✅ Best Practice

expert

Best practice for scaling stateful machine learning services

Which approach is best when scaling a stateful machine learning service that stores user session data?

A Store session data only in pod memory to improve speed.

B Scale pods freely without session management, relying on pod restarts.

C Use sticky sessions with a load balancer to keep users on the same pod.

D Disable scaling to avoid data loss.

💡 Hint

Think about how to keep user data consistent when scaling.

Sticky sessions ensure users connect to the same pod, preserving session data and avoiding data loss during scaling.

Practice

1. Why do systems need different scaling strategies as they grow?

easy

A. Because all systems grow at the same speed

B. Because scaling always means adding more machines

C. Because different growth patterns require different resource management

D. Because vertical scaling is always better than horizontal scaling

Solution

1. Step 1: Understand system growth patterns

   Systems grow in different ways, such as more users or more data, which affects resource needs differently.

2. Step 2: Match scaling strategy to growth type

   Different growth types require different scaling approaches to manage resources efficiently and keep performance.

3. Final Answer:

   Because different growth patterns require different resource management -> Option C

4. Quick Check:

   Growth patterns = Different strategies [OK]

Hint: Match scaling to how system grows for best results [OK]

Common Mistakes:

• Assuming one scaling method fits all
• Thinking scaling always means adding machines
• Ignoring resource limits of single machines

2. Which of the following is the correct way to describe vertical scaling?

easy

A. Adding more machines to handle more load

B. Making a single machine more powerful by adding CPU or RAM

C. Splitting data across multiple databases

D. Reducing the number of users on the system

Solution

1. Step 1: Define vertical scaling

   Vertical scaling means improving one machine's capacity by adding resources like CPU or memory.

2. Step 2: Compare options

   Making a single machine more powerful by adding CPU or RAM matches this definition; others describe horizontal scaling or unrelated actions.

3. Final Answer:

   Making a single machine more powerful by adding CPU or RAM -> Option B

4. Quick Check:

   Vertical scaling = stronger single machine [OK]

Hint: Vertical scaling = upgrade one machine's power [OK]

Common Mistakes:

• Confusing vertical with horizontal scaling
• Thinking vertical scaling means adding machines
• Selecting unrelated options like reducing users

3. Consider a system that uses horizontal scaling by adding identical servers behind a load balancer. What is the main benefit of this approach?

medium

A. It allows the system to handle more users by distributing load

B. It simplifies the software by using only one server

C. It reduces the need for network connections

D. It increases the power of a single server

Solution

1. Step 1: Understand horizontal scaling

   Horizontal scaling adds more servers to share the workload, improving capacity.

2. Step 2: Identify benefit of load balancing

   Load balancers distribute user requests across servers, allowing more users to be served efficiently.

3. Final Answer:

   It allows the system to handle more users by distributing load -> Option A

4. Quick Check:

   Horizontal scaling = distribute load [OK]

Hint: More servers = more users handled [OK]

Common Mistakes:

• Thinking horizontal scaling powers one server
• Believing it reduces network needs
• Assuming it simplifies software to one server

4. A team tried to scale their ML model serving by only upgrading the CPU and RAM of one server, but the system still slowed down under heavy user load. What is the likely problem?

medium

A. They must have a bug in the model code

B. They needed to reduce the model size instead

C. They should have used a faster programming language

D. They should have added more servers instead of upgrading one

Solution

1. Step 1: Analyze the scaling approach

   Upgrading one server is vertical scaling, which has limits and may not handle very high loads.

2. Step 2: Identify better scaling strategy

   Adding more servers (horizontal scaling) distributes load and improves performance under heavy use.

3. Final Answer:

   They should have added more servers instead of upgrading one -> Option D

4. Quick Check:

   Heavy load needs horizontal scaling [OK]

Hint: Heavy load? Add servers, not just power [OK]

Common Mistakes:

• Blaming model size without checking scaling
• Assuming programming language causes slowdown
• Ignoring scaling limits of single server

5. You manage an ML system that processes large datasets and serves predictions to many users. Vertical scaling is costly and limited. Which combined strategy best balances cost, performance, and reliability?

hard

A. Use horizontal scaling with multiple servers and optimize model efficiency

B. Only upgrade the biggest server continuously

C. Reduce the number of users to fit one server

D. Switch to a simpler model without scaling

Solution

1. Step 1: Evaluate vertical scaling limits

   Vertical scaling is costly and hits hardware limits, so relying on it alone is not sustainable.

2. Step 2: Combine horizontal scaling and optimization

   Adding servers (horizontal scaling) spreads load, while optimizing the model reduces resource use, balancing cost and performance.

3. Step 3: Consider reliability

   Multiple servers improve fault tolerance, making the system more reliable than a single powerful server.

4. Final Answer:

   Use horizontal scaling with multiple servers and optimize model efficiency -> Option A

5. Quick Check:

   Combine horizontal scaling + optimization = best balance [OK]

Hint: Combine adding servers with model optimization [OK]

Common Mistakes:

• Relying only on vertical scaling
• Ignoring user demand growth
• Choosing to reduce users instead of scaling
• Dropping scaling for simpler models only