Rack awareness in HDFS in Hadoop - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is rack awareness in HDFS?

Rack awareness is a feature in HDFS that helps the system know which data nodes are located on which racks. This helps HDFS place data blocks in a way that improves fault tolerance and network bandwidth usage.

beginner

Why does HDFS use rack awareness when storing data?

HDFS uses rack awareness to reduce the risk of data loss by storing copies of data blocks on different racks. This way, if one rack fails, data is still available from another rack.

intermediate

How does rack awareness improve network efficiency in HDFS?

By knowing the rack location of data nodes, HDFS can place replicas to minimize cross-rack network traffic. It prefers to read data from the same rack to reduce network congestion and improve speed.

intermediate

What happens if rack awareness is not configured in HDFS?

If rack awareness is not configured, HDFS treats all nodes as if they are on the same rack. This can lead to placing all replicas on the same rack, increasing risk of data loss if that rack fails.

advanced

How does HDFS decide where to place replicas using rack awareness?

HDFS places one replica on the local node, another on a different node in a different rack, and the third on a different node in the same rack as the first replica. This balances fault tolerance and network efficiency.

What is the main purpose of rack awareness in HDFS?

A To encrypt data for security

B To speed up data processing by using GPUs

C To improve fault tolerance by spreading replicas across racks

D To compress data blocks for storage

Rack awareness helps HDFS place replicas on different racks to protect against rack failure.

If rack awareness is not set, what risk increases in HDFS?

A Data will be duplicated too many times

B Data will be encrypted incorrectly

C Data nodes will run out of memory

D Data blocks may be lost if a rack fails

Without rack awareness, replicas may be placed on the same rack, risking data loss if that rack fails.

How many racks does HDFS try to use when placing three replicas of a block?

A Two racks

B One rack only

C Three racks

D Four racks

HDFS places one replica on the local node, one on a different rack, and the third on the same rack as the first replica.

Which of these is NOT a benefit of rack awareness?

A Reduced cross-rack network traffic

B Faster data encryption

C Improved fault tolerance

D Better data locality

Rack awareness does not affect encryption speed; it focuses on data placement and network efficiency.

What information does HDFS need to implement rack awareness?

A Mapping of data nodes to racks

B CPU speed of each node

C Disk size of each node

D User access permissions

HDFS requires a mapping of which nodes belong to which racks to place replicas properly.

Explain how rack awareness helps improve fault tolerance and network efficiency in HDFS.

Think about how spreading data copies across racks helps keep data safe and speeds up access.

Describe what could happen if rack awareness is not configured in an HDFS cluster.

Consider the risks of not knowing where nodes are physically located.