Batch vs real-time ingestion in Hadoop - Practice Questions

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Batch Ingestion Timing

Which statement best describes when batch ingestion processes data in a Hadoop ecosystem?

A Data is processed continuously as it arrives in small chunks.

B Data is processed immediately upon arrival without any delay.

C Data is processed only when a user requests it in real-time.

D Data is collected over a period and processed all at once at scheduled intervals.

💡 Hint

Think about how batch jobs usually run on a schedule, not instantly.

Batch ingestion collects data over time and processes it in bulk at set times, unlike real-time ingestion which processes data instantly.

❓ Predict Output

intermediate

Output of a Real-time Data Stream Simulation

Given the following pseudo-code simulating real-time ingestion in Hadoop streaming, what will be the output after processing 3 data points?

Hadoop

stream = ['data1', 'data2', 'data3']
processed = []
for item in stream:
    processed.append(item.upper())
print(processed)

A ['DATA1', 'DATA2', 'DATA3']

B ['data1', 'data2', 'data3']

C ['Data1', 'Data2', 'Data3']

D Error: 'upper' method not found

💡 Hint

Check what the upper() method does to strings.

The upper() method converts all characters in a string to uppercase, so each data point becomes uppercase.

❓ data_output

advanced

Data Volume Comparison in Batch vs Real-time

Consider a Hadoop system where batch ingestion processes 10,000 records every hour, and real-time ingestion processes 100 records every minute. How many records does each process handle in 3 hours?

A Batch: 10,000 records; Real-time: 6,000 records

B Batch: 30,000 records; Real-time: 18,000 records

C Batch: 30,000 records; Real-time: 300 records

D Batch: 3,000 records; Real-time: 1,800 records

💡 Hint

Calculate total records by multiplying rate by time for both methods.

Batch: 10,000 records/hour * 3 hours = 30,000 records.
Real-time: 100 records/minute * 60 minutes/hour * 3 hours = 18,000 records.

🔧 Debug

advanced

Identifying Error in Real-time Data Processing Code

What error will the following Hadoop streaming code produce?

data_stream = ['a', 'b', 'c']
result = []
for d in data_stream:
    result.append(d / 2)
print(result)

A TypeError: unsupported operand type(s) for /: 'str' and 'int'

B SyntaxError: invalid syntax

C IndexError: list index out of range

D No error; output will be ['a', 'b', 'c']

💡 Hint

Consider what happens when dividing a string by a number.

You cannot divide a string by an integer, so Python raises a TypeError.

🚀 Application

expert

Choosing Ingestion Method for Time-sensitive Data

You manage a Hadoop system that collects sensor data. The sensors send data every second, and you need to detect anomalies within 5 seconds of data arrival. Which ingestion method is best to meet this requirement?

A Batch ingestion, processing data every hour to reduce system load.

B Batch ingestion, processing data once a day for thorough analysis.

C Real-time ingestion, processing data as it arrives to detect anomalies quickly.

D Neither; manual data checks are better for anomaly detection.

💡 Hint

Think about how quickly you need to respond to data to detect anomalies.

Real-time ingestion processes data immediately, allowing anomaly detection within seconds, which fits the 5-second requirement.