Recall & Review
beginner
What is the role of the Driver in Spark architecture?
The Driver is the main program that controls the Spark application. It converts user code into tasks, schedules them, and collects results.
Click to reveal answer
beginner
What do Executors do in Spark?
Executors run on worker nodes and execute the tasks assigned by the Driver. They also store data in memory or disk for fast access.
Click to reveal answer
beginner
What is the purpose of the Cluster Manager in Spark?
The Cluster Manager allocates resources across the cluster. It manages where Executors run and how many resources they get.
Click to reveal answer
intermediate
How do Driver, Executors, and Cluster Manager work together in Spark?
The Driver sends tasks to Executors. The Cluster Manager assigns resources and starts Executors on worker nodes. Executors run tasks and send results back to the Driver.
Click to reveal answer
beginner
Name two popular Cluster Managers used with Spark.
Two popular Cluster Managers are YARN (Yet Another Resource Negotiator) and Apache Mesos.
Click to reveal answer
What component in Spark is responsible for scheduling tasks?
✗ Incorrect
The Driver schedules tasks and coordinates the execution.
Where do Executors run in a Spark cluster?
✗ Incorrect
Executors run on worker nodes to execute tasks.
Which Spark component manages resource allocation?
✗ Incorrect
The Cluster Manager handles resource allocation across the cluster.
What happens if the Driver fails during a Spark job?
✗ Incorrect
The Driver controls the job; if it fails, the job stops.
Which of these is NOT a Cluster Manager for Spark?
✗ Incorrect
HDFS is a storage system, not a Cluster Manager.
Explain the roles of Driver, Executors, and Cluster Manager in Spark architecture.
Think about who plans, who works, and who manages resources.
You got /3 concepts.
Describe how Spark components interact during a job execution.
Focus on the flow of tasks and resource management.
You got /3 concepts.