Migration from Hadoop to cloud-native - Mini Project: Build & Apply

Migration from Hadoop to Cloud-Native Data Processing

📖 Scenario: You work in a company that currently uses Hadoop to process large data sets. The company wants to move to a cloud-native solution for better scalability and cost efficiency.Your task is to simulate a simple data migration process by creating a dataset, setting a migration threshold, filtering data based on this threshold, and then displaying the filtered data.

🎯 Goal: Build a simple data migration script that filters Hadoop data entries based on a cloud migration readiness score.

📋 What You'll Learn

Create a dictionary with Hadoop data entries and their readiness scores

Set a migration threshold score

Filter the data entries that meet or exceed the threshold

Print the filtered data entries

💡 Why This Matters

🌍 Real World

Companies often need to decide which parts of their big data stored in Hadoop are ready to move to cloud-native platforms for better performance and cost savings.

💼 Career

Data engineers and data scientists use filtering and data selection techniques to prepare data for migration and processing in modern cloud environments.

1

Create Hadoop data readiness dictionary

Create a dictionary called hadoop_data with these exact entries: 'dataset1': 75, 'dataset2': 60, 'dataset3': 85, 'dataset4': 50, 'dataset5': 90

Hadoop

# Create the hadoop_data dictionary with the specified entries
# Your code here

Need a hint?

Use curly braces to create a dictionary with keys as dataset names and values as readiness scores.

2

Set migration threshold

Create a variable called threshold and set it to 70 to represent the minimum readiness score for migration.

Hadoop

hadoop_data = {'dataset1': 75, 'dataset2': 60, 'dataset3': 85, 'dataset4': 50, 'dataset5': 90}
# Set the migration threshold to 70
# Your code here

Need a hint?

Just assign the number 70 to the variable named threshold.

3

Filter datasets ready for migration

Create a new dictionary called ready_for_migration that includes only the entries from hadoop_data where the readiness score is greater than or equal to threshold. Use a dictionary comprehension with dataset and score as variables.

Hadoop

hadoop_data = {'dataset1': 75, 'dataset2': 60, 'dataset3': 85, 'dataset4': 50, 'dataset5': 90}
threshold = 70
# Filter datasets with score >= threshold
# Your code here

Need a hint?

Use a dictionary comprehension with for dataset, score in hadoop_data.items() and an if condition.

4

Display datasets ready for migration

Print the ready_for_migration dictionary to show which datasets are ready to move to the cloud.

Hadoop

hadoop_data = {'dataset1': 75, 'dataset2': 60, 'dataset3': 85, 'dataset4': 50, 'dataset5': 90}
threshold = 70
ready_for_migration = {dataset: score for dataset, score in hadoop_data.items() if score >= threshold}
# Print the ready_for_migration dictionary
# Your code here

Need a hint?

Use print(ready_for_migration) to display the filtered dictionary.