HDFS read and write operations in Hadoop - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is HDFS in Hadoop?

HDFS stands for Hadoop Distributed File System. It stores large data across many computers in a cluster, making data reliable and fast to access.

beginner

How does HDFS write operation work?

When writing data, HDFS splits the file into blocks and stores copies (replicas) of each block on different computers to keep data safe.

beginner

What happens during an HDFS read operation?

HDFS reads data by locating the blocks of the file and fetching them from the computers that store the replicas, usually choosing the closest one to speed up reading.

intermediate

Why does HDFS store multiple replicas of data blocks?

HDFS stores multiple copies of data blocks to protect against computer failures. If one copy is lost, others can be used to keep data safe and available.

intermediate

What role does the NameNode play in HDFS read and write operations?

The NameNode keeps track of where all data blocks are stored. It tells the system where to write new blocks and where to find blocks when reading files.

What does HDFS do when writing a large file?

A Stores the entire file on a single node

B Splits the file into blocks and stores replicas on different nodes

C Compresses the file and stores it on one node

D Sends the file to the client without storing

HDFS splits large files into blocks and stores multiple replicas on different nodes to ensure reliability and fault tolerance.

During a read operation, how does HDFS choose which replica to read?

A Chooses the closest replica to the client

B Randomly picks any replica

C Always reads from the first replica stored

D Reads from all replicas simultaneously

HDFS tries to read from the closest replica to reduce network delay and speed up data access.

What is the main function of the NameNode in HDFS?

A Stores actual data blocks

B Processes data computations

C Handles network communication

D Manages metadata and block locations

The NameNode manages metadata like file names and block locations but does not store the actual data blocks.

Why does HDFS replicate data blocks?

A To increase storage space

B To speed up writing only

C To improve data reliability and fault tolerance

D To reduce network traffic

Replication ensures data is safe even if some nodes fail, improving reliability and fault tolerance.

Which component in HDFS directs clients where to write or read data?

A NameNode

B ResourceManager

C JobTracker

D DataNode

The NameNode tells clients where data blocks are stored for reading or where to write new blocks.

Explain the process of writing a file to HDFS, including how data is split and stored.

Think about how HDFS breaks down big files and keeps copies safe.

Describe how HDFS reads a file and how it ensures fast and reliable access.

Consider how HDFS finds and retrieves data efficiently.