Backup and disaster recovery in Hadoop - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Hadoop Backup Strategies

Which of the following best describes the primary purpose of Hadoop's DistCp tool in backup and disaster recovery?

A It compresses Hadoop data files to save storage space during backup.

B It copies large amounts of data efficiently between Hadoop clusters for backup or migration.

C It monitors Hadoop cluster health to prevent data loss.

D It encrypts Hadoop data before backup to ensure security.

💡 Hint

Think about tools designed to move data between clusters.

DistCp (Distributed Copy) is designed to copy large datasets efficiently across Hadoop clusters, which is essential for backup and disaster recovery.

❓ Predict Output

intermediate

HDFS Snapshot Output

What will be the output of the following HDFS command sequence?

hdfs dfs -mkdir /data
hdfs dfs -put file1.txt /data/
hdfs dfs -createSnapshot /data snap1
hdfs dfs -rm /data/file1.txt
hdfs dfs -ls /data/.snapshot/snap1

A Lists all files except file1.txt in the snapshot.

B Shows an empty directory because file1.txt was deleted.

C Lists file1.txt inside the snapshot directory.

D Returns an error because snapshots cannot be created on /data.

💡 Hint

Remember what snapshots do in HDFS.

HDFS snapshots preserve the state of the directory at the time of snapshot creation. Even if file1.txt is deleted later, it remains visible inside the snapshot.

❓ data_output

advanced

Analyzing Backup Data Size Reduction

You run a Hadoop backup job that uses compression. The original data size is 500 GB. After backup, the compressed backup size is 150 GB. What is the compression ratio?

A 3.33

B 0.3

C 350

D 0.03

💡 Hint

Compression ratio = original size / compressed size.

The compression ratio is calculated by dividing the original data size by the compressed size: 500 / 150 = 3.33.

🔧 Debug

advanced

Identifying Error in Hadoop Backup Script

Consider this Hadoop backup script snippet:

hdfs dfs -mkdir /backup
hadoop distcp /user/data /backup/data_backup
hdfs dfs -rm -r /user/data

What is the main risk or error in this script?

A The rm command will fail because /user/data is not empty.

B The distcp command syntax is incorrect and will cause a syntax error.

C The backup directory /backup does not exist before copying.

D The script deletes original data immediately after copying without verifying backup success.

💡 Hint

Think about safe backup practices.

Deleting original data immediately after copying without checking if the backup succeeded risks data loss if the copy fails.

🚀 Application

expert

Designing a Disaster Recovery Plan for Hadoop

You are tasked with designing a disaster recovery plan for a Hadoop cluster that must minimize downtime and data loss. Which combination of strategies is best?

A Use HDFS snapshots regularly and replicate data to a remote cluster using DistCp.

B Only rely on HDFS replication factor set to 3 within the same cluster.

C Backup data manually once a month and store on local disks.

D Use a single-node cluster backup to external USB drives weekly.

💡 Hint

Consider both data safety and recovery speed.

Combining HDFS snapshots for quick recovery and remote replication with DistCp ensures minimal downtime and protects against cluster-wide failures.