Container allocation in Hadoop - Mini Project: Build & Apply

Container allocation

📖 Scenario: You work in a data center that uses Hadoop to run big data jobs. Each job needs containers to run tasks. Containers are like small rooms where tasks work. You want to manage how many containers each job gets based on available resources.

🎯 Goal: Build a simple program to allocate containers to jobs based on their requested containers and the total containers available.

📋 What You'll Learn

Create a dictionary called job_requests with job names as keys and requested container counts as values.

Create a variable called total_containers to store the total containers available.

Write a loop to allocate containers to each job without exceeding total_containers.

Print the final allocation dictionary called container_allocation.

💡 Why This Matters

🌍 Real World

In Hadoop clusters, managing container allocation helps run multiple jobs efficiently without overloading resources.

💼 Career

Understanding container allocation is important for data engineers and system administrators managing big data workloads.

1

Create job container requests

Create a dictionary called job_requests with these exact entries: 'job1': 4, 'job2': 2, 'job3': 5.

Hadoop

# Create the job_requests dictionary with exact values
# Your code here

Need a hint?

Use curly braces {} to create a dictionary with keys and values separated by colons.

2

Set total containers available

Create a variable called total_containers and set it to 8.

Hadoop

job_requests = {'job1': 4, 'job2': 2, 'job3': 5}
# Set total_containers to 8
# Your code here

Need a hint?

Just assign the number 8 to the variable total_containers.

3

Allocate containers to jobs

Create an empty dictionary called container_allocation. Then use a for loop with variables job and request to iterate over job_requests.items(). Inside the loop, allocate containers to each job without exceeding total_containers. Update container_allocation[job] with the allocated containers and reduce total_containers accordingly.

Hadoop

job_requests = {'job1': 4, 'job2': 2, 'job3': 5}
total_containers = 8
# Create container_allocation dictionary and allocate containers in a for loop
# Your code here

Need a hint?

Use min() to allocate the smaller of requested or available containers. Update the allocation dictionary and subtract allocated containers from total_containers.

4

Print container allocation

Write a print statement to display the container_allocation dictionary.

Hadoop

job_requests = {'job1': 4, 'job2': 2, 'job3': 5}
total_containers = 8
container_allocation = {}
for job, request in job_requests.items():
    allocated = min(request, total_containers)
    container_allocation[job] = allocated
    total_containers -= allocated
# Print the container_allocation dictionary
# Your code here

Need a hint?

Use print(container_allocation) to show the final allocation.