Map, filter, and flatMap operations in Apache Spark - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of map operation on RDD

What is the output of the following Spark code snippet?

Apache Spark

rdd = sc.parallelize([1, 2, 3, 4])
result = rdd.map(lambda x: x * 2).collect()
print(result)

A [2, 4, 6, 8]

B [1, 2, 3, 4]

C [1, 4, 9, 16]

D [4, 8, 12, 16]

💡 Hint

Remember that map applies the function to each element individually.

The map operation multiplies each element by 2, so the output list is each original element doubled.

❓ Predict Output

intermediate

Output of filter operation on RDD

What will be the output of this Spark code?

Apache Spark

rdd = sc.parallelize([10, 15, 20, 25, 30])
result = rdd.filter(lambda x: x > 20).collect()
print(result)

A [15, 25]

B [20, 25, 30]

C [10, 15, 20]

D [25, 30]

💡 Hint

Filter keeps elements where the condition is True.

Only elements greater than 20 are kept, which are 25 and 30.

❓ data_output

advanced

Result of flatMap operation on RDD

Given the following Spark code, what is the output of result.collect()?

Apache Spark

rdd = sc.parallelize(['apple,banana', 'orange', 'grape,melon'])
result = rdd.flatMap(lambda x: x.split(',')).collect()
print(result)

A ['apple banana', 'orange', 'grape melon']

B ['apple,banana', 'orange', 'grape,melon']

C ['apple', 'banana', 'orange', 'grape', 'melon']

D ['apple', 'orange', 'grape']

💡 Hint

flatMap splits each string by comma and flattens the result.

flatMap applies split(',') to each element and flattens the lists into one list of fruits.

❓ visualization

advanced

Visualizing filter and map transformations

You have an RDD with numbers from 1 to 5. You apply filter to keep even numbers, then map to square them. Which bar chart correctly shows the final RDD values?

Apache Spark

rdd = sc.parallelize([1, 2, 3, 4, 5])
filtered = rdd.filter(lambda x: x % 2 == 0)
mapped = filtered.map(lambda x: x ** 2)
result = mapped.collect()

A Bars at 1, 3, 5

B Bars at 4, 16

C Bars at 2, 4

D Bars at 1, 4, 9, 16, 25

💡 Hint

Filter keeps even numbers, map squares them.

Only even numbers 2 and 4 remain after filter, their squares are 4 and 16.

🧠 Conceptual

expert

Understanding difference between map and flatMap

Which statement correctly explains the difference between map and flatMap in Spark?

A map applies a function to each element and returns one output per input; flatMap applies a function that returns a list and flattens all lists into one RDD.

B map applies a function and flattens the result, flatMap applies a function without flattening.

C map can only be used on key-value pairs, flatMap only on lists.

D map and flatMap are identical in Spark and can be used interchangeably.

💡 Hint

Think about the output size relative to input size.

map returns one output per input element; flatMap returns zero or more outputs per input and flattens them.