Hadoop distributions (Cloudera, Hortonworks) - Mini Project: Build & Apply

Explore Hadoop Distributions: Cloudera and Hortonworks

📖 Scenario: You are working as a data engineer in a company that uses Hadoop for big data processing. Your manager wants you to understand the differences between two popular Hadoop distributions: Cloudera and Hortonworks. This knowledge will help you decide which distribution to use for your next project.

🎯 Goal: Build a simple data structure to compare features of Cloudera and Hortonworks distributions, then filter and display key differences.

📋 What You'll Learn

Create a dictionary with Hadoop distributions and their features

Add a configuration variable to select a feature to compare

Use a loop to filter distributions based on the selected feature

Print the filtered results clearly

💡 Why This Matters

🌍 Real World

Understanding Hadoop distributions helps data engineers choose the right tools for big data projects based on features like security and support.

💼 Career

Knowledge of Hadoop distributions is important for roles like data engineer, big data analyst, and system administrator working with Hadoop ecosystems.

1

Create a dictionary of Hadoop distributions and their features

Create a dictionary called hadoop_distributions with these exact entries: 'Cloudera' with features ['Security', 'Management', 'Support'] and 'Hortonworks' with features ['Open Source', 'Security', 'Integration'].

Hadoop

# Create the dictionary hadoop_distributions with the specified entries
# Your code here

Need a hint?

Use curly braces {} to create a dictionary. Each key is a distribution name, and the value is a list of features.

2

Add a feature to filter distributions

Create a variable called feature_to_check and set it to the string 'Security'.

Hadoop

hadoop_distributions = {
    'Cloudera': ['Security', 'Management', 'Support'],
    'Hortonworks': ['Open Source', 'Security', 'Integration']
}
# Create the variable feature_to_check with value 'Security'
# Your code here

Need a hint?

Assign the string 'Security' to the variable feature_to_check.

3

Filter distributions by the selected feature

Create a list called distributions_with_feature that contains the names of distributions from hadoop_distributions where feature_to_check is in their features list. Use a for loop with variables distribution and features to iterate over hadoop_distributions.items().

Hadoop

hadoop_distributions = {
    'Cloudera': ['Security', 'Management', 'Support'],
    'Hortonworks': ['Open Source', 'Security', 'Integration']
}
feature_to_check = 'Security'
# Create the list distributions_with_feature using a for loop
# Your code here

Need a hint?

Use a for loop to check each distribution's features. If the feature matches, add the distribution name to the list.

4

Print the distributions that have the selected feature

Write a print statement to display the text 'Distributions with feature Security:' followed by the list distributions_with_feature.

Hadoop

hadoop_distributions = {
    'Cloudera': ['Security', 'Management', 'Support'],
    'Hortonworks': ['Open Source', 'Security', 'Integration']
}
feature_to_check = 'Security'
distributions_with_feature = []
for distribution, features in hadoop_distributions.items():
    if feature_to_check in features:
        distributions_with_feature.append(distribution)
# Print the distributions_with_feature list with a descriptive message
# Your code here

Need a hint?

Use print with a string and the list variable separated by a comma.