Why Spark replaced MapReduce for big data in Apache Spark - Test Your Understanding

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a Spark session.

Apache Spark

from pyspark.sql import SparkSession
spark = SparkSession.builder.appName([1]).getOrCreate()

Drag options to blanks, or click blank then click option'

AMyApp

BSparkSession

CappName

D"MyApp"

Attempts:

3 left

💡 Hint

Common Mistakes

Forgetting to put quotes around the app name string.

Passing the class name instead of a string.

The appName method requires a string argument to name the Spark application.

2fill in blank

medium

Complete the code to read a CSV file into a Spark DataFrame.

Apache Spark

df = spark.read.format([1]).option("header", "true").load("data.csv")

Drag options to blanks, or click blank then click option'

Acsv

Bjson

Cparquet

Dtext

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'json' or 'parquet' format for a CSV file.

Not specifying the format at all.

To read a CSV file, the format must be set to 'csv'.

3fill in blank

hard

Fix the error in the code to cache a DataFrame in Spark.

Apache Spark

df.[1]()

Drag options to blanks, or click blank then click option'

Astore

Bcache

Cpersist

Dsave

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'persist()' without parameters when 'cache()' is intended.

Using 'save()' which writes data to storage.

The correct method to cache a DataFrame in Spark is 'cache()'.

4fill in blank

hard

Fill both blanks to create a DataFrame with only rows where age is greater than 30.

Apache Spark

filtered_df = df.filter(df.[1] [2] 30)

Drag options to blanks, or click blank then click option'

Aage

B>

C<

Dsalary

Attempts:

3 left

💡 Hint

Common Mistakes

Using the wrong column name like 'salary'.

Using '<' instead of '>'.

To filter rows where age is greater than 30, use df.age > 30.

5fill in blank

hard

Fill all three blanks to create a dictionary comprehension that maps words to their lengths if length is greater than 3.

Apache Spark

lengths = { [1]: [2] for [3] in words if len([3]) > 3 }

Drag options to blanks, or click blank then click option'

Aword

Blen(word)

Ditem

Attempts:

3 left

💡 Hint

Common Mistakes

Using inconsistent variable names.

Not using len(word) for the value.

The comprehension maps each word to its length for words longer than 3 characters.