Node decommissioning and scaling in Hadoop - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is node decommissioning in Hadoop?

Node decommissioning is the process of safely removing a node from a Hadoop cluster without losing data or interrupting running jobs. It ensures data blocks are replicated elsewhere before the node is taken offline.

beginner

Why is node decommissioning important before scaling down a Hadoop cluster?

Decommissioning ensures that data stored on the node is copied to other nodes, preventing data loss. It also allows running tasks to finish or move, maintaining cluster stability during scaling down.

intermediate

What happens during the node decommissioning process in Hadoop?

The node is marked for decommissioning, the cluster replicates its data blocks to other nodes, running tasks are completed or moved, and finally the node is removed from the cluster.

beginner

How does Hadoop handle scaling up the cluster?

Scaling up means adding new nodes to the cluster. Hadoop automatically detects new nodes, balances data across them, and uses them for storage and processing to improve performance.

intermediate

What is the role of the 'exclude' file in Hadoop node decommissioning?

The 'exclude' file lists nodes to be decommissioned. Hadoop reads this file to know which nodes to remove safely from the cluster during decommissioning.

What is the first step in safely removing a node from a Hadoop cluster?

A Delete data on the node

B Turn off the node immediately

C Mark the node for decommissioning

D Add new nodes to the cluster

The first step is to mark the node for decommissioning so Hadoop can start replicating data and moving tasks.

Which file in Hadoop specifies nodes to be decommissioned?

A include file

B exclude file

C config file

D hosts file

The exclude file lists nodes that should be decommissioned from the cluster.

What does scaling up a Hadoop cluster involve?

A Adding nodes

B Deleting data

C Stopping all jobs

D Removing nodes

Scaling up means adding new nodes to increase cluster capacity.

Why must data be replicated before node decommissioning?

A To reduce cluster size

B To speed up the node

C To delete old data

D To prevent data loss

Data replication ensures no data is lost when a node is removed.

What happens to running tasks on a node during decommissioning?

A They finish or move to other nodes

B They are immediately stopped

C They continue running without change

D They are deleted

Running tasks are allowed to finish or are moved to other nodes to maintain job integrity.

Explain the process and importance of node decommissioning in a Hadoop cluster.

Think about what happens before a node is removed to keep data safe and jobs running.

Describe how scaling up and scaling down work in Hadoop clusters.

Consider what happens when you want to grow or shrink your cluster.