Date and timestamp functions in Apache Spark - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the current_date() function return in Apache Spark?

The current_date() function returns the current date as a DateType value without the time part.

beginner

How do you extract the year from a timestamp column in Spark?

Use the year() function to get the year part from a timestamp or date column.

intermediate

What is the difference between unix_timestamp() and to_timestamp() functions?

unix_timestamp() returns the number of seconds since 1970-01-01 00:00:00 UTC as a long integer, while to_timestamp() converts a string to a timestamp type.

beginner

How can you add 5 days to a date column in Spark?

Use the date_add(dateColumn, 5) function to add 5 days to the date in dateColumn.

beginner

What does the datediff(end, start) function compute?

It calculates the number of days between two dates: end minus start.

Which function returns the current timestamp in Spark?

A current_date()

B current_timestamp()

C unix_timestamp()

D to_date()

current_timestamp() returns the current date and time as a timestamp.

How do you convert a string like '2024-06-01 12:30:00' to a timestamp in Spark?

A date_add('2024-06-01 12:30:00', 1)

B unix_timestamp('2024-06-01 12:30:00')

C to_date('2024-06-01 12:30:00')

D to_timestamp('2024-06-01 12:30:00')

to_timestamp() converts a string to a timestamp type.

What does date_sub(dateColumn, 3) do?

A Subtracts 3 days from dateColumn

B Returns the day of the month

C Adds 3 days to dateColumn

D Converts dateColumn to string

date_sub() subtracts days from a date.

Which function gives the number of seconds since 1970-01-01 00:00:00 UTC?

A year()

B current_date()

C unix_timestamp()

D to_timestamp()

unix_timestamp() returns seconds since epoch.

If you want to find the difference in days between two dates, which function do you use?

A datediff()

B date_add()

C date_sub()

D year()

datediff() calculates days between two dates.

Explain how to convert a string to a timestamp and then extract the year in Apache Spark.

Think about the functions to convert and then extract parts of a date.

Describe how to calculate the number of days between two date columns in Spark.

Focus on the function that measures days difference.