Reading JSON and nested data in Apache Spark - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to read a JSON file into a DataFrame.

Apache Spark

df = spark.read.[1]("data.json")

Drag options to blanks, or click blank then click option'

Ajson

Bparquet

Ctext

Dcsv

Attempts:

3 left

💡 Hint

Common Mistakes

Using csv() or text() to read JSON files causes errors or wrong data.

Forgetting to specify the file path as a string.

The json method reads JSON files into a DataFrame.

2fill in blank

medium

Complete the code to select the nested field 'address.city' from the DataFrame.

Apache Spark

df.select("[1]")

Drag options to blanks, or click blank then click option'

Aaddress

Bcity

Caddress.city

Daddress[city]

Attempts:

3 left

💡 Hint

Common Mistakes

Selecting only 'address' returns the whole nested struct, not the city.

Using brackets like address[city] is not valid syntax here.

Use dot notation to select nested fields in Spark DataFrames.

3fill in blank

hard

Fix the error in the code to explode the nested array field 'phones'.

Apache Spark

from pyspark.sql.functions import [1]
df.select(explode(df.phones)).show()

Drag options to blanks, or click blank then click option'

Aexplode

Bflatten

Ccollect_list

Darray

Attempts:

3 left

💡 Hint

Common Mistakes

Using flatten or collect_list does not explode arrays into rows.

Not importing the function causes a NameError.

The explode function expands an array column into multiple rows.

4fill in blank

hard

Fill both blanks to create a dictionary of word lengths for words longer than 3 characters.

Apache Spark

lengths = {word: [1] for word in words if [2]

Drag options to blanks, or click blank then click option'

Alen(word)

Blen(word) > 3

Cword.startswith('a')

Dword.isalpha()

Attempts:

3 left

💡 Hint

Common Mistakes

Using conditions unrelated to length causes wrong filtering.

Not using len(word) for the dictionary values.

The dictionary comprehension maps each word to its length if the word length is greater than 3.

5fill in blank

hard

Fill all three blanks to create a dictionary with uppercase keys and values greater than 0.

Apache Spark

result = [1]: [2] for k, v in data.items() if v [3] 0}

Drag options to blanks, or click blank then click option'

Ak.upper()

Bv

C>

Dk.lower()

Attempts:

3 left

💡 Hint

Common Mistakes

Using k.lower() instead of uppercase keys.

Using wrong comparison operators like < or ==.

The dictionary comprehension uses uppercase keys, keeps values, and filters values greater than zero.