Migration from Hadoop to cloud-native - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Key advantage of cloud-native over Hadoop

Which of the following is the main advantage of migrating from Hadoop to a cloud-native data platform?

A Cloud-native platforms require manual resource allocation for each job to optimize costs.

B Cloud-native platforms provide automatic scaling and managed services reducing operational overhead.

C Hadoop clusters are easier to set up and require less maintenance than cloud-native platforms.

D Hadoop supports serverless computing natively, unlike cloud-native platforms.

💡 Hint

Think about how cloud platforms handle resources compared to traditional Hadoop clusters.

Cloud-native platforms offer automatic scaling and managed services, which reduce the need for manual cluster management, unlike Hadoop which requires manual setup and maintenance.

❓ data_output

intermediate

Output of data migration script

Given the following Python pseudocode for migrating data from HDFS to cloud storage, what will be the output summary?

Hadoop

files = ['file1.csv', 'file2.csv', 'file3.csv']
migrated = []
for f in files:
    if f.endswith('.csv'):
        migrated.append(f)
print(f'Migrated {len(migrated)} files: {migrated}')

A Migrated 2 files: ['file1.csv', 'file2.csv']

B Migrated 0 files: []

C Migrated 3 files: ['file1.csv', 'file2.csv', 'file3.csv']

D Migrated 3 files: ['file1.csv', 'file2.csv']

💡 Hint

Check which files end with '.csv' in the list.

All three files end with '.csv', so all are appended to the migrated list, resulting in 3 files migrated.

🔧 Debug

advanced

Identify error in cloud migration script

What error will the following Python code raise during migration?

Hadoop

def migrate_data(files):
    migrated = {}
    for f in files:
        migrated[f] = migrated.get(f, 0) + 1
    return migrated

files = ['data1', 'data2', 'data1']
result = migrate_data(files)
print(result['data3'])

A KeyError because 'data3' is not in the dictionary.

B SyntaxError due to missing colon in function definition.

C No error, output will be {'data1': 2, 'data2': 1, 'data3': 0}.

D TypeError because dictionary keys must be integers.

💡 Hint

Check what happens when accessing a dictionary key that does not exist.

Accessing result['data3'] raises a KeyError because 'data3' was never added to the dictionary.

❓ visualization

advanced

Interpreting migration progress chart

A line chart shows the number of files migrated over time during a Hadoop to cloud migration. The line starts slow, then sharply rises, then plateaus. What does this pattern most likely indicate?

A Files were deleted instead of migrated, causing the plateau.

B Constant migration speed with no changes in resource allocation.

C Migration failed early and restarted multiple times.

D Initial setup delays, followed by fast migration, then resource limits causing slowdown.

💡 Hint

Think about common phases in large data migrations and resource usage.

The slow start is typical setup time, the sharp rise shows active migration, and the plateau suggests hitting resource or quota limits.

🚀 Application

expert

Choosing the best cloud-native tool for batch processing

You need to migrate a Hadoop batch processing job to a cloud-native environment. The job processes large datasets with complex transformations and requires fault tolerance. Which cloud-native tool is best suited?

A Apache Spark on managed Kubernetes with autoscaling and checkpointing.

B Simple cloud storage buckets with manual script execution.

C Serverless functions with 1-minute execution limits.

D Relational database triggers for batch processing.

💡 Hint

Consider fault tolerance, scalability, and batch processing capabilities.

Apache Spark on managed Kubernetes supports complex batch jobs, fault tolerance via checkpointing, and autoscaling, making it ideal for migrating Hadoop batch jobs.