SparkSession and SparkContext in Apache Spark - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a SparkSession named spark.

Apache Spark

from pyspark.sql import SparkSession
spark = SparkSession.builder.appName([1]).getOrCreate()

Drag options to blanks, or click blank then click option'

A"MyApp"

BMyApp

Cspark

DSparkSession

Attempts:

3 left

💡 Hint

Common Mistakes

Forgetting to put the app name in quotes.

Using a variable name without quotes.

The appName method requires a string argument with the application name, so it must be in quotes.

2fill in blank

medium

Complete the code to get the SparkContext from the SparkSession.

Apache Spark

sc = spark.[1]

Drag options to blanks, or click blank then click option'

Acontext

BsparkContext

CSparkContext

DgetContext

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'SparkContext' instead of 'sparkContext'.

Trying to call a method instead of accessing an attribute.

The SparkContext is accessed from SparkSession by the attribute sparkContext (camelCase).

3fill in blank

hard

Fix the error in the code to stop the SparkSession properly.

Apache Spark

spark.[1]()

Drag options to blanks, or click blank then click option'

Astop

BstopSession

Cclose

Dend

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'close()' or 'end()' which do not exist.

Trying to call 'stopSession()' which is incorrect.

The correct method to stop a SparkSession is 'stop()'.

4fill in blank

hard

Fill both blanks to create a SparkSession with master set to local and app name 'TestApp'.

Apache Spark

spark = SparkSession.builder.master([1]).appName([2]).getOrCreate()

Drag options to blanks, or click blank then click option'

A"local"

B"TestApp"

Clocal

DTestApp

Attempts:

3 left

💡 Hint

Common Mistakes

Not using quotes around 'local' or 'TestApp'.

Using unquoted values causing syntax errors.

Both master and appName require string arguments in quotes.

5fill in blank

hard

Fill all three blanks to create a dictionary with word lengths for words longer than 3 characters.

Apache Spark

lengths = { [1]: [2] for [3] in words if len([3]) > 3 }

Drag options to blanks, or click blank then click option'

Aword

Blen(word)

Attempts:

3 left

💡 Hint

Common Mistakes

Using different variable names for key and loop variable.

Not using len(word) for the value.

The dictionary comprehension uses 'word' as key and 'len(word)' as value, iterating over 'word' in words.