What is an RDD (Resilient Distributed Dataset) in Apache Spark - Practice Questions & Exercises

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding RDD Characteristics

Which of the following best describes a Resilient Distributed Dataset (RDD) in Apache Spark?

A A fault-tolerant collection of elements that can be operated on in parallel across a cluster.

B A single-node data storage system optimized for fast reads.

C A graphical user interface for managing Spark jobs.

D A type of database used to store structured data only.

💡 Hint

Think about how Spark handles data across multiple machines and recovers from failures.

An RDD is a fundamental Spark data structure that is distributed across cluster nodes and can recover from node failures automatically, making it fault-tolerant.

🧠 Conceptual

intermediate

RDD Immutability

What does it mean that RDDs are immutable in Apache Spark?

A Once created, an RDD cannot be changed; transformations create new RDDs instead.

B RDDs automatically delete old data after processing.

C RDDs can be updated in place to save memory.

D RDDs allow direct modification of data on cluster nodes.

💡 Hint

Consider how Spark manages data consistency and fault tolerance.

RDDs are immutable, meaning you cannot change their data after creation. Instead, transformations produce new RDDs, which helps with fault tolerance and consistency.

❓ Predict Output

advanced

RDD Transformation Output

What is the output of the following Spark code snippet?

Apache Spark

rdd = sc.parallelize([1, 2, 3, 4, 5])
result = rdd.filter(lambda x: x % 2 == 0).collect()
print(result)

A [1, 2, 3, 4, 5]

B [2, 4]

C [1, 3, 5]

D []

💡 Hint

Filter keeps elements where the condition is true.

The filter keeps only even numbers (divisible by 2), so the output is [2, 4].

❓ data_output

advanced

RDD Action Result

Given the following RDD and action, what is the output?

Apache Spark

rdd = sc.parallelize([10, 20, 30, 40])
sum_value = rdd.reduce(lambda a, b: a + b)
print(sum_value)

A TypeError

B 40

C 10

D 100

💡 Hint

Reduce combines all elements using the given function.

The reduce sums all elements: 10 + 20 + 30 + 40 = 100.

🔧 Debug

expert

Identifying Error in RDD Code

What error will the following Spark code produce?

Apache Spark

rdd = sc.parallelize([1, 2, 3])
result = rdd.map(lambda x: x / 0).collect()
print(result)

A ValueError

B TypeError

C ZeroDivisionError

D No error, outputs [inf, inf, inf]

💡 Hint

Consider what happens when dividing by zero in Python.

Dividing by zero raises a ZeroDivisionError in Python, so the Spark job will fail with this error.