Flume for log collection in Hadoop - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to specify the Flume source type for collecting logs.

Hadoop

agent.sources.src.type = [1]

Drag options to blanks, or click blank then click option'

Aavro

Bexec

Cfile

Dhttp

Attempts:

3 left

💡 Hint

Common Mistakes

Choosing 'file' which is not a valid Flume source type.

Choosing 'avro' which is for Avro RPC source, not file logs.

The exec source type is used in Flume to collect logs by running an external command such as tail -F /path/to/logfile.

2fill in blank

medium

Complete the code to set the Flume sink type to write logs into HDFS.

Hadoop

agent.sinks.sink.type = [1]

Drag options to blanks, or click blank then click option'

Alogger

Bavro

Chdfs

Dfile_roll

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'logger' which only logs to console.

Choosing 'file_roll' which writes to local files, not HDFS.

The hdfs sink writes the collected data into HDFS storage.

3fill in blank

hard

Fix the error in the Flume configuration to correctly bind the source to the channel.

Hadoop

agent.sources.src.channels = [1]

Drag options to blanks, or click blank then click option'

Ach1

BsourceChannel

Cchannel-1

Dchannel1

Attempts:

3 left

💡 Hint

Common Mistakes

Using a channel name that does not match the defined channel.

Including special characters in the channel name.

The channel name must match the channel defined elsewhere. 'channel1' is the correct channel name.

4fill in blank

hard

Fill both blanks to configure the Flume agent's memory channel with capacity and transaction capacity.

Hadoop

agent.channels.memChannel.type = [1]
agent.channels.memChannel.capacity = [2]

Drag options to blanks, or click blank then click option'

Amemory

Bfile

C10000

D5000

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'file' as channel type instead of 'memory'.

Setting capacity to a non-numeric value.

The memory channel type is 'memory'. Capacity is set to 10000 events.

5fill in blank

hard

Fill all three blanks to configure the Flume HDFS sink with path, file prefix, and roll interval.

Hadoop

agent.sinks.hdfsSink.hdfs.path = [1]
agent.sinks.hdfsSink.hdfs.filePrefix = [2]
agent.sinks.hdfsSink.hdfs.rollInterval = [3]

Drag options to blanks, or click blank then click option'

A/logs/flume

Blogfile

C60

D/data/input

Attempts:

3 left

💡 Hint

Common Mistakes

Using a local path instead of HDFS path.

Setting roll interval to zero or negative number.

The HDFS path is '/logs/flume', file prefix is 'logfile', and roll interval is 60 seconds.