Caching and persistence in Apache Spark - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to cache the DataFrame named df.

Apache Spark

df.[1]()

Drag options to blanks, or click blank then click option'

Ashow

Bpersist

Ccollect

Dcache

Attempts:

3 left

💡 Hint

Common Mistakes

Using collect() which brings data to driver but does not cache.

Using show() which only displays data.

Using persist() without specifying storage level.

The cache() method stores the DataFrame in memory for faster access.

2fill in blank

medium

Complete the code to persist the DataFrame df with MEMORY_AND_DISK storage level.

Apache Spark

from pyspark import StorageLevel

df.[1](StorageLevel.MEMORY_AND_DISK)

Drag options to blanks, or click blank then click option'

Apersist

Bcache

Cunpersist

Dcollect

Attempts:

3 left

💡 Hint

Common Mistakes

Using cache() which does not accept storage level argument.

Using unpersist() which removes caching.

Using collect() which gathers data to driver.

The persist() method allows specifying storage level like MEMORY_AND_DISK.

3fill in blank

hard

Fix the error in the code to unpersist the DataFrame df.

Apache Spark

df.[1](blocking=True)

Drag options to blanks, or click blank then click option'

Apersist

Bcache

Cunpersist

Dcollect

Attempts:

3 left

💡 Hint

Common Mistakes

Using cache() or persist() which add caching instead of removing.

Using collect() which does not affect caching.

The unpersist() method removes the cached or persisted data from memory/disk.

4fill in blank

hard

Fill both blanks to cache the DataFrame df and then unpersist it blocking until done.

Apache Spark

df.[1]()
df.[2](blocking=True)

Drag options to blanks, or click blank then click option'

Acache

Bpersist

Cunpersist

Dcollect

Attempts:

3 left

💡 Hint

Common Mistakes

Using persist instead of cache for the first method.

Not using blocking=True in unpersist.

First, cache() stores the DataFrame in memory. Then, unpersist(blocking=True) removes it and waits until done.

5fill in blank

hard

Fill all three blanks to persist df with MEMORY_AND_DISK, then cache it, then unpersist blocking.

Apache Spark

from pyspark import StorageLevel

df.[1](StorageLevel.MEMORY_AND_DISK)
df.[2]()
df.[3](blocking=True)

Drag options to blanks, or click blank then click option'

Apersist

Bcache

Cunpersist

Dcollect

Attempts:

3 left

💡 Hint

Common Mistakes

Using collect instead of unpersist.

Not specifying storage level in persist.

Using cache before persist.

We first persist with MEMORY_AND_DISK, then cache (which is redundant but allowed), then unpersist blocking.