String functions in Spark in Apache Spark - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the Spark function lower() do to a string column?

It converts all characters in the string column to lowercase letters.

beginner

How does the concat() function work in Spark?

It joins two or more string columns or literals into one single string column.

beginner

What is the purpose of the substring() function in Spark?

It extracts a part of a string from a specified start position for a given length.

beginner

Explain what trim() does in Spark string functions.

It removes leading and trailing spaces from the string values in a column.

intermediate

What does the regexp_replace() function do in Spark?

It replaces all substrings that match a regular expression pattern with a specified replacement string.

Which Spark string function converts all letters in a column to uppercase?

A lower()

B substring()

C trim()

D upper()

The upper() function changes all characters to uppercase.

What does the concat() function do in Spark?

A Joins multiple strings into one

B Extracts a substring

C Removes spaces

D Splits a string into parts

concat() joins two or more strings into a single string.

Which function removes spaces at the start and end of a string in Spark?

A rtrim()

B ltrim()

C trim()

D substring()

trim() removes spaces from both ends of the string.

How do you extract a part of a string starting at position 3 with length 4 in Spark?

A substr(column, 3, 4)

B substr(column, 4, 3)

C substring(column, 4, 3)

D substring(column, 3, 4)

substr(column, start, length) extracts substring; start=3, length=4.

What does regexp_replace() do in Spark?

A Finds a substring

B Replaces substrings matching a pattern

C Splits strings by pattern

D Counts pattern matches

regexp_replace() replaces all matches of a regex pattern with a new string.

Describe how you would clean a string column in Spark to remove extra spaces and convert all text to lowercase.

Think about removing spaces and changing case.

Explain how to combine two string columns into one in Spark and then extract a part of the combined string.

First join strings, then get a substring.