Backup and disaster recovery in Hadoop - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a backup directory in HDFS.

Hadoop

hdfs dfs -mkdir [1]

Drag options to blanks, or click blank then click option'

A/user/data

B/input

C/tmp

D/backup

Attempts:

3 left

💡 Hint

Common Mistakes

Using a directory path that already exists or is not intended for backup.

Forgetting the leading slash in the directory path.

The command 'hdfs dfs -mkdir /backup' creates a backup directory in HDFS.

2fill in blank

medium

Complete the code to copy data from HDFS to the backup directory.

Hadoop

hdfs dfs -cp [1] /backup/

Drag options to blanks, or click blank then click option'

A/user/hadoop/data

B/backup

C/tmp/data

D/user/hadoop/logs

Attempts:

3 left

💡 Hint

Common Mistakes

Copying from the backup directory itself.

Using a temporary or unrelated directory as source.

The command copies data from '/user/hadoop/data' to the backup directory '/backup/'.

3fill in blank

hard

Fix the error in the command to restore data from backup to the original location.

Hadoop

hdfs dfs -mv /backup/[1] /user/hadoop/data

Drag options to blanks, or click blank then click option'

Alogs

Bbackup

Cdata

Dtmp

Attempts:

3 left

💡 Hint

Common Mistakes

Using the backup directory name instead of the data folder.

Using unrelated folder names like 'logs' or 'tmp'.

The data folder inside /backup is moved back to the original location '/user/hadoop/data'.

4fill in blank

hard

Fill both blanks to create a snapshot of the /user/hadoop/data directory.

Hadoop

hdfs dfsadmin -allowSnapshot [1]
hdfs dfs -createSnapshot [2] snapshot1

Drag options to blanks, or click blank then click option'

A/user/hadoop/data

B/backup

D/tmp

Attempts:

3 left

💡 Hint

Common Mistakes

Allowing snapshots on one directory but creating snapshot on another.

Using backup or temporary directories instead of the data directory.

First, snapshots are allowed on the data directory, then a snapshot named 'snapshot1' is created there.

5fill in blank

hard

Fill all three blanks to list snapshots, delete a snapshot, and verify deletion.

Hadoop

hdfs dfs -ls [1]
hdfs dfs -deleteSnapshot [2] [3]
hdfs dfs -ls [1]

Drag options to blanks, or click blank then click option'

A/user/hadoop/data

Csnapshot1

Dsnapshot2

Attempts:

3 left

💡 Hint

Common Mistakes

Using different directories for listing and deleting snapshots.

Deleting a snapshot that does not exist.

The commands list snapshots in the data directory, delete 'snapshot1', then list snapshots again to confirm deletion.

Practice

1. What is the main purpose of backup in Hadoop's disaster recovery?

easy

A. To delete old data automatically

B. To save copies of data to prevent loss

C. To speed up data processing

D. To encrypt data for security

Solution

1. Step 1: Understand backup purpose

   Backup means saving copies of data so it can be recovered if lost.

2. Step 2: Match with options

   Only To save copies of data to prevent loss describes saving copies to prevent data loss.

3. Final Answer:

   To save copies of data to prevent loss -> Option B

4. Quick Check:

   Backup = Save copies [OK]

Hint: Backup means saving data copies to avoid loss [OK]

Common Mistakes:

• Confusing backup with data processing speed
• Thinking backup deletes data
• Mixing backup with encryption

2. Which Hadoop command is used to copy files from HDFS to local for backup?

easy

A. hdfs dfs -get

B. hdfs dfs -ls

C. hdfs dfs -rm

D. hdfs dfs -put

Solution

1. Step 1: Identify command to copy from HDFS to local

   The command hdfs dfs -get copies files from HDFS to local filesystem.

2. Step 2: Check other commands

   -put copies local to HDFS, -rm removes files, -ls lists files.

3. Final Answer:

   hdfs dfs -get -> Option A

4. Quick Check:

   Copy HDFS to local = -get [OK]

Hint: Use -get to copy from HDFS to local [OK]

Common Mistakes:

• Using -put instead of -get
• Confusing -rm with copy commands
• Thinking -ls copies files

3. What will be the output of this command if the source file does not exist?
hdfs dfs -cp /data/file1.txt /backup/file1.txt

medium

A. cp: `/data/file1.txt`: No such file or directory

B. File copied successfully

C. Backup created with empty file

D. Command runs silently without error

Solution

1. Step 1: Understand the cp command behavior

   The hdfs dfs -cp command copies files within HDFS. If the source file does not exist, it throws an error.

2. Step 2: Match error message

   The error message for missing source file is "No such file or directory" as in cp: `/data/file1.txt`: No such file or directory.

3. Final Answer:

   cp: `/data/file1.txt`: No such file or directory -> Option A

4. Quick Check:

   Missing source file = error message [OK]

Hint: Missing source file causes 'No such file' error [OK]

Common Mistakes:

• Assuming silent success
• Expecting empty file creation
• Confusing error with copy success

4. Identify the error in this backup command:
hdfs dfs -copyToLocal /backup/data.txt /local/backup/

medium

A. The command should be -copyFromLocal to backup

B. The destination path should be HDFS, not local

C. The command is correct as is

D. The source path should be local, not HDFS

Solution

1. Step 1: Understand -copyToLocal command

   This command copies files from HDFS to local filesystem, useful for backup.

2. Step 2: Check source and destination paths

   Source is HDFS (/backup/data.txt), destination is local (/local/backup/), which matches command purpose.

3. Final Answer:

   The command is correct as is -> Option C

4. Quick Check:

   copyToLocal copies HDFS to local [OK]

Hint: copyToLocal copies from HDFS to local correctly [OK]

Common Mistakes:

• Confusing copyToLocal with copyFromLocal
• Swapping source and destination paths
• Thinking backup requires copyFromLocal

5. You want to create a disaster recovery plan that backs up daily changes in Hadoop data to a remote cluster. Which approach is best?

hard

A. Use hdfs dfs -rm to remove data after backup

B. Manually copy files with hdfs dfs -cp on local machine

C. Delete old backups and keep only latest file in HDFS

D. Use hdfs dfs -distcp to copy changed files to remote cluster daily

Solution

1. Step 1: Understand disaster recovery needs

   Disaster recovery requires copying data safely to a remote cluster regularly.

2. Step 2: Evaluate commands for remote copying

   hdfs dfs -distcp is designed for efficient distributed copying between clusters, ideal for daily backups.

3. Step 3: Check other options

   Manual copy is error-prone and slow; deleting backups or removing data is risky.

4. Final Answer:

   Use hdfs dfs -distcp to copy changed files to remote cluster daily -> Option D

5. Quick Check:

   distcp = best for remote backup [OK]

Hint: Use distcp for efficient remote cluster backup [OK]

Common Mistakes:

• Using local copy commands for remote backup
• Deleting backups too early
• Confusing remove command with backup