Compute resource management in MLOps - Mini Project: Build & Apply

Compute Resource Management

📖 Scenario: You are managing a small machine learning project that needs to allocate compute resources efficiently. You want to track the available compute nodes and their capacities to decide where to run your training jobs.

🎯 Goal: Build a simple Python program that stores compute nodes and their CPU capacities, sets a minimum CPU threshold, filters nodes that meet this threshold, and prints the eligible nodes.

📋 What You'll Learn

Create a dictionary called compute_nodes with exact node names and CPU counts

Create a variable called min_cpu with the minimum CPU threshold

Use a dictionary comprehension called eligible_nodes to filter nodes with CPUs >= min_cpu

Print the eligible_nodes dictionary

💡 Why This Matters

🌍 Real World

Managing compute resources is essential in machine learning projects to allocate jobs efficiently and avoid overloading nodes.

💼 Career

DevOps and MLOps engineers often write scripts to monitor and manage compute resources to optimize performance and cost.

1

Create the compute nodes dictionary

Create a dictionary called compute_nodes with these exact entries: 'node1': 8, 'node2': 4, 'node3': 16, 'node4': 2

MLOps

# Create the compute_nodes dictionary with exact node names and CPU counts
# Your code here

Need a hint?

Use curly braces {} to create a dictionary with keys as node names and values as CPU counts.

2

Set the minimum CPU threshold

Create a variable called min_cpu and set it to the integer 6

MLOps

compute_nodes = {'node1': 8, 'node2': 4, 'node3': 16, 'node4': 2}
# Create min_cpu variable and set it to 6
# Your code here

Need a hint?

Just assign the number 6 to the variable min_cpu.

3

Filter eligible nodes with enough CPUs

Use a dictionary comprehension called eligible_nodes to include only nodes from compute_nodes where the CPU count is greater than or equal to min_cpu

MLOps

compute_nodes = {'node1': 8, 'node2': 4, 'node3': 16, 'node4': 2}
min_cpu = 6
# Create eligible_nodes dictionary comprehension filtering nodes with CPUs >= min_cpu
# Your code here

Need a hint?

Use {node: cpu for node, cpu in compute_nodes.items() if cpu >= min_cpu} to filter the dictionary.

4

Print the eligible nodes

Write a print statement to display the eligible_nodes dictionary

MLOps

compute_nodes = {'node1': 8, 'node2': 4, 'node3': 16, 'node4': 2}
min_cpu = 6
eligible_nodes = {node: cpu for node, cpu in compute_nodes.items() if cpu >= min_cpu}
# Print the eligible_nodes dictionary
# Your code here

Need a hint?

Use print(eligible_nodes) to show the filtered nodes.