Type casting and null handling in Apache Spark - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to cast the column 'age' to IntegerType.

Apache Spark

df = df.withColumn('age', df['age'].cast([1]))

Drag options to blanks, or click blank then click option'

AIntegerType()

BStringType()

CFloatType()

DBooleanType()

Attempts:

3 left

💡 Hint

Common Mistakes

Using StringType instead of IntegerType causes wrong data type.

Using FloatType when integer is needed.

Forgetting to import IntegerType.

We use IntegerType() to cast the column to integer values.

2fill in blank

medium

Complete the code to replace null values in 'salary' column with 0.

Apache Spark

df = df.fillna({'salary': [1])

Drag options to blanks, or click blank then click option'

A-1

BNone

C''

D0

Attempts:

3 left

💡 Hint

Common Mistakes

Using None does not replace nulls.

Using empty string '' for numeric columns causes errors.

Using negative numbers without reason.

We replace nulls in 'salary' with 0 to avoid null errors in calculations.

3fill in blank

hard

Fix the error in casting 'price' column to DoubleType.

Apache Spark

from pyspark.sql.types import [1]
df = df.withColumn('price', df['price'].cast(DoubleType()))

Drag options to blanks, or click blank then click option'

ADoubleType

BStringType

CIntegerType

DFloatType

Attempts:

3 left

💡 Hint

Common Mistakes

Importing FloatType instead of DoubleType.

Forgetting to import the type causes NameError.

Using IntegerType for decimal numbers.

We must import DoubleType to cast to double precision float.

4fill in blank

hard

Complete the code to fill null values in 'age' with 0 and 'name' with 'Unknown' using a dictionary.

Apache Spark

df = df.fillna({'age': [1], 'name': [2])

Drag options to blanks, or click blank then click option'

A0

B'Unknown'

CNone

D''

Attempts:

3 left

💡 Hint

Common Mistakes

Using a string like '0' for the numeric 'age' column.

Forgetting single quotes around 'Unknown'.

Using the same value for both numeric and string columns.

We use a dictionary in fillna() to specify different replacement values for different columns: 0 for numeric 'age' and 'Unknown' for string 'name'.

5fill in blank

hard

Fill all three blanks to conditionally replace nulls in 'salary' with 0 using when and isNull, then cast to DoubleType.

Apache Spark

df = df.withColumn('salary_clean', when(df['salary'].[1](), [2]).otherwise(df['salary']).[3](DoubleType()))

Drag options to blanks, or click blank then click option'

AisNull

Blit(0)

Ccast

Dfillna

Attempts:

3 left

💡 Hint

Common Mistakes

Using == 'null' or == None instead of isNull().

Forgetting lit() around the replacement value.

Applying cast() before handling nulls.

Using wrong method like fillna in when clause.

Use isNull() to detect nulls, lit(0) as replacement literal, and cast(DoubleType()) to ensure double type.