Why transformations build processing pipelines in Apache Spark - Test Your Understanding

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a new RDD by applying a transformation.

Apache Spark

rdd2 = rdd.[1](lambda x: x * 2)

Drag options to blanks, or click blank then click option'

Afilter

Breduce

Ccollect

Dmap

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'collect' which returns data to the driver, not a transformation.

Using 'reduce' which aggregates data, not element-wise transformation.

The map transformation applies a function to each element and returns a new RDD.

2fill in blank

medium

Complete the code to filter RDD elements greater than 10.

Apache Spark

filtered_rdd = rdd.[1](lambda x: x > 10)

Drag options to blanks, or click blank then click option'

Amap

BflatMap

Cfilter

DreduceByKey

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'map' which transforms all elements but does not filter.

Using 'reduceByKey' which is for key-value RDDs.

The filter transformation keeps only elements that satisfy the condition.

3fill in blank

hard

Fix the error in the code to chain transformations correctly.

Apache Spark

result = rdd.[1](lambda x: x + 1).[2](lambda x: x * 2)

Drag options to blanks, or click blank then click option'

Amap

Bfilter

Creduce

Dcollect

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'filter' in the second step which expects a boolean function.

Using 'collect' which triggers action, not transformation.

Both transformations should be map to apply functions element-wise and chain properly.

4fill in blank

hard

Fill both blanks to build a pipeline that filters and then maps the RDD.

Apache Spark

pipeline = rdd.[1](lambda x: x % 2 == 0).[2](lambda x: x * 10)

Drag options to blanks, or click blank then click option'

Afilter

Bmap

Creduce

DflatMap

Attempts:

3 left

💡 Hint

Common Mistakes

Reversing the order of transformations.

Using 'reduce' which aggregates instead of transforming.

First filter keeps even numbers, then map multiplies them by 10.

5fill in blank

hard

Fill all three blanks to create a pipeline that filters, maps, and then collects results.

Apache Spark

final_result = rdd.[1](lambda x: x > 5).[2](lambda x: x - 1).[3]()

Drag options to blanks, or click blank then click option'

Afilter

Bmap

Ccollect

Dreduce

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'reduce' instead of 'collect' which returns a single value.

Calling 'collect' before transformations.

The pipeline filters elements greater than 5, maps by subtracting 1, then collects the results to the driver.