MapReduce job execution flow in Hadoop - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the first step in the MapReduce job execution flow?

The first step is Job Submission, where the client submits the job to the Hadoop cluster, including the code and input data location.

intermediate

What role does the JobTracker play in MapReduce execution?

The JobTracker manages the job by splitting it into tasks, assigning them to TaskTrackers, and monitoring their progress.

beginner

Explain the function of the Mapper in MapReduce.

The Mapper processes input data splits and produces intermediate key-value pairs for further processing.

intermediate

What happens during the Shuffle and Sort phase?

Intermediate data from Mappers is shuffled and sorted by keys to prepare for the Reducer, ensuring all values for a key are grouped together.

beginner

Describe the final step in the MapReduce job execution flow.

The final step is Job Completion, where the Reducers write output to HDFS, and the JobTracker reports job status back to the client.

What component assigns tasks to nodes in a Hadoop cluster?

A NameNode

B JobTracker

C DataNode

D ResourceManager

The JobTracker is responsible for assigning MapReduce tasks to TaskTrackers (nodes).

During which phase are intermediate key-value pairs grouped by key?

A Job submission

B Map phase

C Reduce phase

D Shuffle and Sort phase

The Shuffle and Sort phase groups intermediate data by key before reducing.

What does the Mapper output in MapReduce?

A Intermediate key-value pairs

B Input splits

C Job status

D Final results

Mappers output intermediate key-value pairs for the Reducers.

Where is the final output of a MapReduce job stored?

A HDFS

B Local disk

C JobTracker memory

D Mapper output buffer

The final output is stored in the Hadoop Distributed File System (HDFS).

Which phase immediately follows the Map phase?

A Job submission

B Reduce phase

C Shuffle and Sort phase

D Job completion

After mapping, data is shuffled and sorted before reducing.

Describe the complete flow of a MapReduce job from submission to completion.

Think about how data moves and changes from start to finish in the cluster.

Explain the purpose of the Shuffle and Sort phase in MapReduce.

Consider why Reducers need data organized before processing.