What is an RDD (Resilient Distributed Dataset) in Apache Spark - Interactive Quiz & Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create an RDD from a list in Spark.

Apache Spark

rdd = sparkContext.[1]([1, 2, 3, 4])

Drag options to blanks, or click blank then click option'

Aparallelize

Bcollect

Cmap

Dfilter

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'collect' which gathers data back to the driver, not creating an RDD.

Using 'map' or 'filter' which are transformations, not creation methods.

The parallelize method creates an RDD from a local collection like a list.

2fill in blank

medium

Complete the code to apply a transformation that doubles each number in the RDD.

Apache Spark

doubled_rdd = rdd.[1](lambda x: x * 2)

Drag options to blanks, or click blank then click option'

Amap

Bfilter

Creduce

Dcollect

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'filter' which removes elements instead of transforming them.

Using 'reduce' which combines elements into one value.

The map transformation applies a function to each element in the RDD.

3fill in blank

hard

Fix the error in the code to collect the results from the RDD.

Apache Spark

results = rdd.[1]()

Drag options to blanks, or click blank then click option'

Areduce

Bcollect

Cfilter

Dmap

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'map' which returns a new RDD, not the data itself.

Using 'reduce' which combines elements into a single value.

The collect action returns all elements of the RDD to the driver as a list.

4fill in blank

hard

Fill both blanks to create an RDD and filter out numbers less than 5.

Apache Spark

filtered_rdd = sparkContext.[1]([3, 6, 1, 8]).[2](lambda x: x >= 5)

Drag options to blanks, or click blank then click option'

Aparallelize

Bmap

Cfilter

Dcollect

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'map' instead of 'filter' to remove elements.

Using 'collect' too early which brings data back to the driver.

First, parallelize creates the RDD. Then, filter keeps only numbers 5 or greater.

5fill in blank

hard

Fill all three blanks to create an RDD, transform it by adding 10, and collect the results.

Apache Spark

result = sparkContext.[1]([1, 2, 3]).[2](lambda x: x + 10).[3]()

Drag options to blanks, or click blank then click option'

Aparallelize

Bmap

Ccollect

Dfilter

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'filter' instead of 'map' for transformation.

Forgetting to collect results to see output.

First, parallelize creates the RDD. Then, map adds 10 to each element. Finally, collect gathers the results to the driver.