Reading from Kafka with Spark in Apache Spark - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to read data from Kafka using Spark.

Apache Spark

df = spark.readStream.format([1]).option("kafka.bootstrap.servers", "localhost:9092").option("subscribe", "topic1").load()

Drag options to blanks, or click blank then click option'

A"kafka"

B"json"

C"csv"

D"parquet"

Attempts:

3 left

💡 Hint

Common Mistakes

Using formats like "json" or "csv" instead of "kafka".

Forgetting to specify the Kafka bootstrap servers.

To read from Kafka, the format must be set to "kafka" in Spark.

2fill in blank

medium

Complete the code to select the Kafka message value as a string.

Apache Spark

messages = df.selectExpr("CAST(value AS [1]) AS value")

Drag options to blanks, or click blank then click option'

A"string"

B"boolean"

C"float"

D"int"

Attempts:

3 left

💡 Hint

Common Mistakes

Casting to numeric types like "int" or "float" without conversion.

Not casting the value at all.

Kafka message values are binary, so casting to "string" makes them readable text.

3fill in blank

hard

Fix the error in the code to write streaming data back to Kafka.

Apache Spark

query = messages.writeStream.format([1]).option("kafka.bootstrap.servers", "localhost:9092").option("topic", "output_topic").start()

Drag options to blanks, or click blank then click option'

A"json"

B"console"

C"kafka"

D"parquet"

Attempts:

3 left

💡 Hint

Common Mistakes

Using "console" or "json" as the sink format when writing to Kafka.

Forgetting to specify the Kafka bootstrap servers.

To write streaming data to Kafka, the format must be "kafka".

4fill in blank

hard

Fill both blanks to filter messages with value length greater than 5.

Apache Spark

filtered = messages.filter(length([1]) [2] 5)

Drag options to blanks, or click blank then click option'

A"value"

B>

C<

D"key"

Attempts:

3 left

💡 Hint

Common Mistakes

Filtering on "key" instead of "value".

Using the wrong comparison operator.

We filter on the "value" column where its length is greater than 5.

5fill in blank

hard

Fill all three blanks to create a dictionary of word counts from Kafka messages.

Apache Spark

word_counts = {word: [1] for word in words if [2](word) [3] 3}

Drag options to blanks, or click blank then click option'

Alen(word)

Blen

C>

Dcount

Attempts:

3 left

💡 Hint

Common Mistakes

Using the wrong variable in the condition.

Using incorrect comparison operators.

We count the length of each word and include only words longer than 3 characters.