Hadoop distributions (Cloudera, Hortonworks) - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to show the command that starts the Cloudera Manager service.

Hadoop

sudo systemctl [1] cloudera-scm-server

Drag options to blanks, or click blank then click option'

Astart

Bstop

Crestart

Dstatus

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'stop' instead of 'start'.

Using 'status' which only shows service status.

The command sudo systemctl start cloudera-scm-server starts the Cloudera Manager service.

2fill in blank

medium

Complete the code to list all services managed by Hortonworks Ambari.

Hadoop

ambari-server [1]

Drag options to blanks, or click blank then click option'

Alist-services

Bstatus

Cstop

Dstart

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'status' which shows Ambari server status only.

Using 'start' or 'stop' which control the server.

The command ambari-server list-services lists all services managed by Ambari.

3fill in blank

hard

Fix the error in the command to check the status of the Hadoop service in Cloudera Manager.

Hadoop

sudo service hadoop-[1] status

Drag options to blanks, or click blank then click option'

Aresourcemanager

Bnamenode

Cdatanode

Dnodemanager

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'datanode' which is a worker node service.

Using 'resourcemanager' which manages resources, not filesystem.

The correct service name to check the Hadoop NameNode status is hadoop-namenode.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that maps service names to their status in Ambari.

Hadoop

service_status = { [1]: ambari_client.get_status([2]) for [1] in services_list }

Drag options to blanks, or click blank then click option'

Aservice

Cstatus

Dservices_list

Attempts:

3 left

💡 Hint

Common Mistakes

Using different variable names for key and function argument.

Using 'status' or 'services_list' as the key.

The dictionary comprehension uses service as the key and calls ambari_client.get_status(service) for each service in services_list.

5fill in blank

hard

Fill all three blanks to filter and create a dictionary of Cloudera services with status 'running'.

Hadoop

running_services = { [1]: [2] for [1] in all_services if [2] == 'running' }

Drag options to blanks, or click blank then click option'

Aservice

Bstatus

Cservice_status[service]

Dall_services

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing variable names for keys and values.

Using the wrong variable in the if condition.

The comprehension iterates over all_services using service as key, gets the status from service_status[service], and filters where service_status[service] == 'running'.

Practice

1. What is the main purpose of Hadoop distributions like Cloudera and Hortonworks?

easy

A. To provide only storage without any processing capabilities

B. To replace Hadoop with a completely different big data system

C. To bundle Hadoop with extra tools and support for easier big data management

D. To limit Hadoop usage to small datasets only

Solution

1. Step 1: Understand what Hadoop distributions include

   Hadoop distributions package Hadoop core with additional tools and management features.

2. Step 2: Identify the purpose of these added tools

   These tools help users manage big data projects more easily and reliably.

3. Final Answer:

   To bundle Hadoop with extra tools and support for easier big data management -> Option C

4. Quick Check:

   Distributions = Hadoop + tools + support [OK]

Hint: Distributions add tools and support to Hadoop [OK]

Common Mistakes:

• Thinking distributions replace Hadoop
• Believing distributions only provide storage
• Assuming distributions limit data size

2. Which of the following is the correct way to describe Cloudera in the context of Hadoop?

easy

A. A Hadoop distribution that includes management and security tools

B. A programming language used to write Hadoop jobs

C. A cloud service unrelated to Hadoop

D. A database system that replaces Hadoop

Solution

1. Step 1: Identify Cloudera's role

   Cloudera is known as a Hadoop distribution with extra management and security features.

2. Step 2: Eliminate incorrect options

   It is not a programming language, cloud service unrelated to Hadoop, or a database replacing Hadoop.

3. Final Answer:

   A Hadoop distribution that includes management and security tools -> Option A

4. Quick Check:

   Cloudera = Hadoop distribution + tools [OK]

Hint: Cloudera bundles Hadoop with management tools [OK]

Common Mistakes:

• Confusing Cloudera with a programming language
• Thinking Cloudera is unrelated to Hadoop
• Assuming Cloudera replaces Hadoop

3. Given the following code snippet in a Hadoop environment using Hortonworks distribution, what will be the output?

from hdfs import InsecureClient
client = InsecureClient('http://localhost:50070')
files = client.list('/user/hadoop')
print(len(files))

medium

A. A syntax error due to missing import

B. The number of files in the /user/hadoop directory

C. An empty list printed

D. A runtime error because the client URL is incorrect

Solution

1. Step 1: Understand the code functionality

   The code connects to HDFS via Hortonworks client and lists files in /user/hadoop directory.

2. Step 2: Analyze the output of print(len(files))

   It prints the count of files found in that directory, which is the length of the list returned.

3. Final Answer:

   The number of files in the /user/hadoop directory -> Option B

4. Quick Check:

   len(files) = file count [OK]

Hint: list() returns files; len() counts them [OK]

Common Mistakes:

• Confusing list() output with file contents
• Expecting syntax error without cause
• Assuming client URL is always wrong

4. Identify the error in this Hortonworks Hadoop configuration snippet:

hdfs-site.xml content:
<configuration>
  <property>
    <name>dfs.replication</name>
    <value>one</value>
  </property>
</configuration>

medium

A. The replication value should be a number, not the word 'one'

B. The property name dfs.replication is incorrect

C. The XML tags are not closed properly

D. The configuration file should be named core-site.xml

Solution

1. Step 1: Check the dfs.replication value type

   The dfs.replication property expects a numeric value indicating replication factor.

2. Step 2: Identify the error in the value

   The value 'one' is a string and invalid; it should be a number like 1.

3. Final Answer:

   The replication value should be a number, not the word 'one' -> Option A

4. Quick Check:

   dfs.replication = numeric value [OK]

Hint: Replication factor must be numeric [OK]

Common Mistakes:

• Using words instead of numbers for replication
• Confusing property names
• Misnaming configuration files

5. You want to migrate a big data project from Hortonworks to Cloudera. Which of the following steps is MOST important to ensure a smooth transition?

hard

A. Rewrite all Hadoop MapReduce jobs in a new programming language

B. Ignore configuration differences since both are Hadoop

C. Delete all data before migration to avoid conflicts

D. Verify compatibility of data formats and replication settings between distributions

Solution

1. Step 1: Understand migration challenges

   Different Hadoop distributions may have subtle differences in data formats and replication configurations.

2. Step 2: Identify key step for smooth migration

   Verifying compatibility of these settings prevents data loss or errors during migration.

3. Final Answer:

   Verify compatibility of data formats and replication settings between distributions -> Option D

4. Quick Check:

   Check compatibility before migration [OK]

Hint: Check data and config compatibility before migrating [OK]

Common Mistakes:

• Thinking rewriting code is always needed
• Deleting data unnecessarily
• Ignoring config differences