Kafka integration with Hadoop - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to specify the Kafka topic to consume data from.

Hadoop

kafkaParams = {"metadata.broker.list": "localhost:9092"}
topic = "[1]"

rdd = KafkaUtils.createDirectStream(ssc, [topic], kafkaParams)

Drag options to blanks, or click blank then click option'

Akafka_topic

Bdata_stream

Chadoop_topic

Dinput_topic

Attempts:

3 left

💡 Hint

Common Mistakes

Using an incorrect or non-existent topic name.

Leaving the topic name empty or misspelled.

The Kafka topic name must be set correctly to consume data. Here, "kafka_topic" is the correct topic name used in the example.

2fill in blank

medium

Complete the code to set the Kafka broker list for Hadoop integration.

Hadoop

kafkaParams = {"metadata.broker.list": "[1]"}

rdd = KafkaUtils.createDirectStream(ssc, ["kafka_topic"], kafkaParams)

Drag options to blanks, or click blank then click option'

Alocalhost:9092

Blocalhost:2181

C127.0.0.1:8080

Dkafka:9092

Attempts:

3 left

💡 Hint

Common Mistakes

Using the Zookeeper port (2181) instead of Kafka broker port.

Using incorrect host or port format.

The Kafka broker list specifies the host and port where Kafka brokers listen. The default Kafka port is 9092.

3fill in blank

hard

Fix the error in the code to correctly create a direct Kafka stream in Spark Streaming.

Hadoop

from pyspark.streaming.kafka import KafkaUtils

rdd = KafkaUtils.createDirectStream(ssc, ["kafka_topic"], [1])

Drag options to blanks, or click blank then click option'

AkafkaParams()

Bkafka_params

CkafkaParams

DkafkaParams[]

Attempts:

3 left

💡 Hint

Common Mistakes

Calling the dictionary as a function.

Using incorrect variable name casing.

The parameter must be the dictionary variable 'kafkaParams' without parentheses or brackets.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that filters Kafka messages with value length greater than 5.

Hadoop

filtered_messages = {msg.key: msg.value for msg in messages if len(msg.[1]) [2] 5}

Drag options to blanks, or click blank then click option'

Avalue

B>

C<

Dkey

Attempts:

3 left

💡 Hint

Common Mistakes

Filtering by key length instead of value length.

Using '<' instead of '>' causing wrong filtering.

We check the length of the message value and filter those with length greater than 5.

5fill in blank

hard

Fill all three blanks to create a dictionary comprehension that maps uppercase keys to values where value length is less than 10.

Hadoop

result = {msg.[1].upper(): msg.[2] for msg in messages if len(msg.[3]) < 10}

Drag options to blanks, or click blank then click option'

Akey

Bvalue

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing up keys and values in comprehension.

Not converting keys to uppercase.

The keys are converted to uppercase, values are mapped directly, and filtering is done on value length less than 10.