Rack awareness in HDFS in Hadoop - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why is rack awareness important in HDFS?

Rack awareness helps HDFS decide where to place data blocks. What is the main benefit of using rack awareness?

A It reduces network traffic by placing replicas on different racks to avoid data loss if a rack fails.

B It speeds up data processing by storing all replicas on the same rack for faster access.

C It simplifies the file system by ignoring network topology during data placement.

D It increases storage capacity by duplicating data on the same node multiple times.

💡 Hint

Think about what happens if a whole rack goes down.

Rack awareness places replicas on different racks to protect against rack failure and reduce cross-rack network traffic.

🧠 Conceptual

intermediate

How does HDFS use rack awareness during block placement?

When HDFS writes a block, how does rack awareness influence where replicas are stored?

A All replicas are stored on the same node to maximize speed.

B One replica is stored on the local node, another on a different node in the same rack, and the third on a node in a different rack.

C Replicas are randomly placed without considering racks.

D Two replicas are stored on the same rack and one on a different data center.

💡 Hint

Think about balancing fault tolerance and network traffic.

HDFS places one replica locally, one on the same rack, and one on a different rack to balance reliability and network efficiency.

❓ data_output

advanced

Output of rack-aware block placement simulation

Given the following simplified Python simulation of HDFS rack-aware block placement, what is the output?

Hadoop

nodes = {'rack1': ['node1', 'node2'], 'rack2': ['node3', 'node4']}

import random

# Place one replica on node1
replicas = ['node1']

# Place second replica on different node in same rack
same_rack_nodes = [n for n in nodes['rack1'] if n != 'node1']
replicas.append(random.choice(same_rack_nodes))

# Place third replica on a node in a different rack
other_rack = 'rack2'
replicas.append(random.choice(nodes[other_rack]))

print(sorted(replicas))

A ['node2', 'node3', 'node4']

B ['node1', 'node2', 'node1']

C ['node1', 'node2', 'node3'] or ['node1', 'node2', 'node4']

D ['node1', 'node3', 'node4']

💡 Hint

Remember the second replica is on the same rack but different node, third on different rack.

The code places replicas on node1, another node in rack1 (node2), and one node in rack2 (node3 or node4). The output is sorted so node1 and node2 appear first, then node3 or node4.

🔧 Debug

advanced

Identify the error in rack-aware replica placement code

What error will this Python code raise when simulating rack-aware replica placement?

nodes = {'rack1': ['node1', 'node2'], 'rack2': ['node3', 'node4']}

replicas = []

# Place one replica on node1
replicas.append('node1')

# Place second replica on different node in same rack
same_rack_nodes = [n for n in nodes['rack1'] if n != 'node1']
replicas.append(same_rack_nodes[1])

# Place third replica on a node in a different rack
other_rack = 'rack2'
replicas.append(nodes[other_rack][0])

print(replicas)

A No error, code runs and prints the replica list.

B KeyError because 'rack3' does not exist in nodes dictionary.

C TypeError because replicas list cannot append strings.

D IndexError because same_rack_nodes has only 1 element but index 2 is accessed.

💡 Hint

Check the length of same_rack_nodes before accessing index 2.

same_rack_nodes contains only one node ('node2'), so accessing index 2 causes IndexError.

🚀 Application

expert

Analyzing network traffic reduction using rack awareness

Suppose an HDFS cluster has 3 racks with 4 nodes each. Without rack awareness, replicas are placed randomly. With rack awareness, replicas are placed as per the standard policy (one local, one same rack, one different rack). Which option best describes the expected impact on cross-rack network traffic during data reads?

A Rack awareness reduces cross-rack traffic by ensuring at least two replicas are on the same rack, minimizing data transfer across racks.

B Rack awareness has no effect on network traffic since data is always read from the closest node regardless of rack.

C Rack awareness increases cross-rack traffic because replicas are spread across all racks evenly.

D Rack awareness causes all replicas to be on the same rack, eliminating cross-rack traffic completely.

💡 Hint

Think about how placing replicas on the same rack affects network usage.

By placing one replica locally and another on the same rack, rack awareness reduces the need to fetch data across racks, lowering cross-rack network traffic.