Spot instances for cost savings in Apache Spark - Cheat Sheet & Quick Revision

Recall & Review

beginner

What are spot instances in cloud computing?

Spot instances are spare cloud computing resources offered at a lower price, but they can be interrupted by the cloud provider when demand increases.

beginner

How do spot instances help reduce costs in Apache Spark workloads?

Spot instances allow running Apache Spark tasks on cheaper resources, lowering overall compute costs, especially for flexible or fault-tolerant workloads.

intermediate

What is a key risk when using spot instances for Spark jobs?

The main risk is that spot instances can be terminated unexpectedly, which may cause Spark jobs to fail or require restarting.

intermediate

Name one strategy to handle spot instance interruptions in Spark clusters.

One strategy is to use Spark's checkpointing and task retries to recover from interruptions without losing all progress.

beginner

Why are spot instances suitable for batch processing jobs in Spark?

Because batch jobs can tolerate delays or restarts, spot instances provide a cost-effective way to run large-scale Spark batch workloads.

What happens to spot instances when cloud demand increases?

A They are terminated or reclaimed by the cloud provider

B They become more expensive

C They run without interruption

D They automatically upgrade to on-demand instances

Spot instances can be interrupted or terminated when demand rises, as they use spare capacity.

Which Spark workload is best suited for spot instances?

A Batch processing jobs that can tolerate restarts

B Long-running jobs without checkpointing

C Interactive queries requiring immediate results

D Real-time streaming with strict latency

Batch jobs that can handle interruptions are ideal for spot instances due to cost savings.

What feature in Spark helps recover from spot instance interruptions?

A Disabling fault tolerance

B Static resource allocation

C Manual job restarts only

D Checkpointing and task retries

Checkpointing saves progress, and retries allow tasks to restart after interruptions.

Why might spot instances be cheaper than on-demand instances?

A They use older hardware

B They are spare capacity offered at a discount

C They have fewer features

D They are slower

Spot instances use spare cloud capacity and are priced lower to encourage usage.

What is a common approach to minimize data loss when using spot instances in Spark?

A Avoid saving intermediate results

B Run jobs only on on-demand instances

C Use checkpointing to save state periodically

D Disable retries

Checkpointing helps save progress so jobs can resume after interruptions.

Explain how spot instances can reduce costs for Apache Spark workloads and what risks they introduce.

Think about why spot instances are cheaper and what happens when they are taken away.

Describe strategies to handle spot instance interruptions when running Spark jobs.

Consider how Spark can save progress and recover from failures.