How to Decommission a Node in Hadoop Cluster

To decommission a node in Hadoop, add the node's hostname to the dfs.exclude file and update the configuration with dfs.hosts.exclude. Then run hdfs dfsadmin -refreshNodes to start decommissioning, which safely moves data off the node before removal.

Syntax

Decommissioning a Hadoop node involves these key steps:

• dfs.exclude: A file listing nodes to remove.
• dfs.hosts.exclude: Configuration property pointing to the exclude file.
• hdfs dfsadmin -refreshNodes: Command to apply changes and start decommissioning.

bash

echo "node-to-decommission-hostname" >> /etc/hadoop/conf/dfs.exclude
hdfs dfsadmin -refreshNodes

Example

This example shows how to decommission a node named node3.example.com safely from the Hadoop cluster.

bash

# Add the node to the exclude file
echo "node3.example.com" >> /etc/hadoop/conf/dfs.exclude

# Ensure the configuration points to the exclude file
# In hdfs-site.xml, dfs.hosts.exclude should be set to /etc/hadoop/conf/dfs.exclude

# Refresh nodes to start decommissioning
hdfs dfsadmin -refreshNodes

# Check decommission status
hdfs dfsadmin -report

Output

Configured Capacity: 1000 GB
DFS Remaining: 800 GB
Decommission Status : Decommissioning node3.example.com
...

Common Pitfalls

• Not updating dfs.hosts.exclude to point to the exclude file causes no effect.
• Forgetting to run hdfs dfsadmin -refreshNodes means the cluster ignores the exclude list.
• Removing the node before decommission completes can cause data loss.
• Not monitoring the decommission status can lead to premature node shutdown.

bash

## Wrong way: Forgetting to refresh nodes

echo "node3.example.com" >> /etc/hadoop/conf/dfs.exclude
# Missing: hdfs dfsadmin -refreshNodes

## Right way:
hdfs dfsadmin -refreshNodes

Quick Reference

Step	Command/Config	Description
1	Add node to dfs.exclude file	List nodes to decommission
2	Set dfs.hosts.exclude in hdfs-site.xml	Point to dfs.exclude file
3	hdfs dfsadmin -refreshNodes	Apply changes and start decommission
4	hdfs dfsadmin -report	Check decommission status
5	Shutdown node after decommission completes	Safely remove node

Key Takeaways

Add nodes to be decommissioned in the dfs.exclude file.

Set dfs.hosts.exclude property to point to the exclude file in hdfs-site.xml.

Run hdfs dfsadmin -refreshNodes to start the decommission process.

Monitor decommission status with hdfs dfsadmin -report before shutting down the node.

Never remove a node before decommission completes to avoid data loss.