Cost optimization at scale in MLOps - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Spot Instances for Cost Savings

Which statement best describes the main advantage of using spot instances in cloud-based machine learning workflows?

A Spot instances offer lower prices but can be interrupted, making them suitable for fault-tolerant batch training jobs.

B Spot instances provide guaranteed uptime and are ideal for critical real-time inference tasks.

C Spot instances are more expensive but provide better GPU performance than on-demand instances.

D Spot instances automatically scale the number of CPUs based on workload without user intervention.

💡 Hint

Think about cost versus reliability trade-offs in cloud compute options.

Spot instances are cheaper but can be stopped by the cloud provider at any time. They are best for jobs that can handle interruptions, like batch training, not for critical real-time tasks.

💻 Command Output

intermediate

Analyzing Cost with Kubectl Metrics

Given the command kubectl top pods --namespace=ml-training, what output will you see?

MLOps

kubectl top pods --namespace=ml-training

A A list of pods with their CPU and memory usage in the ml-training namespace.

B A list of all namespaces with their total CPU and memory usage.

C An error stating 'metrics API not available' if metrics-server is not installed.

D A list of nodes with their CPU and memory usage.

💡 Hint

Consider what 'top pods' command shows in Kubernetes.

The command shows resource usage (CPU and memory) for pods in the specified namespace. If metrics-server is missing, it errors out.

🔀 Workflow

advanced

Optimizing Model Training with Autoscaling

You want to reduce costs by automatically scaling your training cluster based on GPU usage. Which Kubernetes resource should you configure?

A Configure a Cluster Autoscaler to add or remove nodes based on GPU resource requests.

B Configure a Vertical Pod Autoscaler (VPA) to increase pod memory limits automatically.

C Configure a Horizontal Pod Autoscaler (HPA) targeting CPU usage metrics only.

D Configure a DaemonSet to run GPU monitoring agents on each node.

💡 Hint

Think about scaling nodes, not just pods, when GPU resources are involved.

Cluster Autoscaler adjusts the number of nodes in the cluster based on resource requests like GPUs, helping optimize costs by adding/removing nodes as needed.

❓ Troubleshoot

advanced

Identifying Cost Spikes in MLOps Pipelines

Your cloud bill suddenly increased after deploying a new ML pipeline. Which tool can help you identify which pipeline step caused the cost spike?

A Use Docker logs to check container output for errors.

B Use Prometheus to monitor CPU and memory usage metrics during pipeline execution.

C Use Git to track changes in pipeline code that might increase costs.

D Use a cloud provider's cost explorer to analyze spending by resource tags assigned to pipeline steps.

💡 Hint

Think about tools that analyze spending by resource usage or tags.

Cost explorer tools let you break down cloud spending by tags or resources, helping pinpoint which pipeline step caused the spike.

✅ Best Practice

expert

Implementing Cost Controls in MLOps

Which practice best helps prevent unexpected high costs in a large-scale MLOps environment?

A Allow unrestricted access to cloud resources for all team members to speed up development.

B Set up budget alerts and enforce resource quotas per team or project.

C Disable autoscaling to keep resource usage constant and predictable.

D Use only on-demand instances to avoid interruptions.

💡 Hint

Think about proactive cost management and limits.

Budget alerts notify teams when spending approaches limits, and resource quotas prevent teams from using more resources than allowed, controlling costs effectively.