Lazy evaluation in Spark in Apache Spark - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Lazy Evaluation in Spark

Which statement best describes lazy evaluation in Apache Spark?

A Spark delays execution of transformations until an action is called.

B Spark immediately executes all transformations as soon as they are called.

C Spark executes transformations in parallel without waiting for actions.

D Spark caches all data automatically to speed up computations.

💡 Hint

Think about when Spark actually runs the computations.

Lazy evaluation means Spark waits to run transformations until an action triggers execution.

❓ Predict Output

intermediate

Output of Spark Transformations Without Action

What will be the output of the following Spark code snippet?

rdd = sc.parallelize([1, 2, 3, 4])
mapped_rdd = rdd.map(lambda x: x * 2)
print(mapped_rdd.collect())

Apache Spark

rdd = sc.parallelize([1, 2, 3, 4])
mapped_rdd = rdd.map(lambda x: x * 2)
print(mapped_rdd.collect())

A [2, 4, 6, 8]

B SyntaxError due to missing action

C An empty list []

D None

💡 Hint

Consider what triggers execution in Spark.

The collect() action triggers execution, so the doubled values are returned.

❓ data_output

advanced

Number of Jobs Triggered by Actions

Given the following Spark code, how many Spark jobs will be triggered?

rdd = sc.parallelize([1, 2, 3, 4])
mapped = rdd.map(lambda x: x + 1)
filtered = mapped.filter(lambda x: x % 2 == 0)
count = filtered.count()
collected = filtered.collect()

Apache Spark

rdd = sc.parallelize([1, 2, 3, 4])
mapped = rdd.map(lambda x: x + 1)
filtered = mapped.filter(lambda x: x % 2 == 0)
count = filtered.count()
collected = filtered.collect()

A 1

B 2

C 0

D 3

💡 Hint

Each action triggers a job. How many actions are there?

There are two actions: count() and collect(), so two jobs run.

🔧 Debug

advanced

Identifying the Cause of No Output

Why does the following Spark code produce no output?

rdd = sc.parallelize([10, 20, 30])
rdd.map(lambda x: x * 3)

Apache Spark

rdd = sc.parallelize([10, 20, 30])
rdd.map(lambda x: x * 3)

A The RDD is empty so no output is produced.

B The lambda function syntax is incorrect causing silent failure.

C The map transformation is lazy and no action was called to trigger execution.

D Spark requires caching before transformations to produce output.

💡 Hint

Think about what triggers Spark to run transformations.

Transformations like map are lazy and need an action like collect() to run.

🚀 Application

expert

Optimizing Spark Job Execution

You have a Spark job with multiple transformations and two actions on the same RDD. How can you optimize to avoid running the same transformations twice?

A Call <code>collect()</code> after each transformation to save intermediate results.

B Rewrite the transformations as actions to force immediate execution.

C Split the RDD into two separate RDDs to run actions independently.

D Use <code>cache()</code> or <code>persist()</code> on the RDD before the actions to reuse computed data.

💡 Hint

Think about how Spark can reuse data between actions.

Caching or persisting stores the RDD in memory or disk to avoid recomputing transformations for multiple actions.