Batch vs real-time ingestion in Hadoop - Interactive Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to read a batch file from HDFS.

Hadoop

hadoop fs -cat /data/batch/[1]

Drag options to blanks, or click blank then click option'

Ainput.txt

Bstreaming

Crealtime

Dlive

Attempts:

3 left

💡 Hint

Common Mistakes

Using streaming or realtime keywords instead of a file name.

Confusing batch with real-time ingestion.

The batch ingestion reads static files like 'input.txt' from HDFS.

2fill in blank

medium

Complete the code to start a real-time data stream using Flume.

Hadoop

flume-ng agent --conf conf --conf-file [1] --name agent1 -Dflume.root.logger=INFO,console

Drag options to blanks, or click blank then click option'

Adata.csv

Bbatch.conf

Cinput.txt

Dstream.conf

Attempts:

3 left

💡 Hint

Common Mistakes

Using batch configuration file for real-time ingestion.

Using data files instead of config files.

Real-time ingestion uses a streaming configuration file like 'stream.conf' for Flume.

3fill in blank

hard

Fix the error in the command to run a batch MapReduce job.

Hadoop

hadoop jar wordcount.jar WordCount [1] /output

Drag options to blanks, or click blank then click option'

A/output

B/input/batch

C/input/streaming

D/tmp

Attempts:

3 left

💡 Hint

Common Mistakes

Using output directory as input path.

Using streaming input path for batch job.

Batch jobs read from batch input directories like '/input/batch'.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that filters batch files larger than 100MB.

Hadoop

large_files = {file: size for file, size in files.items() if size [1] [2]

Drag options to blanks, or click blank then click option'

A>

B100

C<

D50

Attempts:

3 left

💡 Hint

Common Mistakes

Using '<' instead of '>' causing wrong filtering.

Using wrong size threshold.

The comprehension filters files with size greater than 100MB.

5fill in blank

hard

Fill all three blanks to create a dictionary of real-time streams with their status if active.

Hadoop

active_streams = {stream[1]: status for stream, status in streams.items() if status [2] 'active' and stream.startswith([3])}

Drag options to blanks, or click blank then click option'

A.upper()

B==

C'rt_'

D!=

Attempts:

3 left

💡 Hint

Common Mistakes

Using '!=' instead of '==' causing wrong filtering.

Not converting stream names to uppercase.

Using wrong prefix string.

The code filters streams with status 'active' and names starting with 'rt_', converting stream names to uppercase as keys.