Why data format affects performance in Apache Spark - Test Your Understanding

Practice - 5 Tasks

1fill in blank

easy

Complete the code to read a Parquet file into a Spark DataFrame.

Apache Spark

df = spark.read.[1]("data.parquet")

Drag options to blanks, or click blank then click option'

Aparquet

Bcsv

Cjson

Dtext

Attempts:

3 left

💡 Hint

Common Mistakes

Using csv or json to read a parquet file causes errors or slow performance.

Using text format reads data as plain text, losing structure.

Parquet is a columnar storage format optimized for Spark. Using parquet reads the file correctly and efficiently.

2fill in blank

medium

Complete the code to write a DataFrame in an efficient columnar format.

Apache Spark

df.write.[1]("output_path")

Drag options to blanks, or click blank then click option'

Aparquet

Bcsv

Cjson

Dtext

Attempts:

3 left

💡 Hint

Common Mistakes

Writing as csv or json can slow down later reads.

Text format loses data structure and is inefficient.

Writing data in Parquet format stores it in a columnar way, improving read and query speed.

3fill in blank

hard

Fix the error in the code to read a JSON file with Spark.

Apache Spark

df = spark.read.[1]("data.json")

Drag options to blanks, or click blank then click option'

Aparquet

Bcsv

Ctext

Djson

Attempts:

3 left

💡 Hint

Common Mistakes

Using parquet or csv to read JSON files causes errors.

Using text reads raw strings, losing JSON structure.

To read a JSON file, use spark.read.json(). Using other formats causes errors or wrong parsing.

4fill in blank

hard

Fill both blanks to create a dictionary of word lengths for words longer than 3 characters.

Apache Spark

lengths = {word: [1] for word in words if len(word) [2] 3}

Drag options to blanks, or click blank then click option'

Alen(word)

B<=

C>

Dword

Attempts:

3 left

💡 Hint

Common Mistakes

Using the word itself as value instead of its length.

Using <= instead of > in the condition.

The dictionary maps each word to its length using len(word). The condition filters words longer than 3 with >.

5fill in blank

hard

Fill all three blanks to create a dictionary with uppercase keys and values greater than zero.

Apache Spark

result = { [1]: [2] for k, v in data.items() if v [3] 0 }

Drag options to blanks, or click blank then click option'

Ak.upper()

Bv

C>

Dk

Attempts:

3 left

💡 Hint

Common Mistakes

Using original keys instead of uppercase.

Using <= or < instead of > in the condition.

Swapping keys and values.

The dictionary uses uppercase keys with k.upper(), values are v, and filters values greater than zero with >.