GroupBy and aggregations in Apache Spark - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the groupBy() function do in Apache Spark?

It groups rows in a DataFrame based on one or more columns, so you can perform aggregations on each group separately.

beginner

Name three common aggregation functions used after groupBy() in Spark.

Common aggregation functions are count(), sum(), and avg() (average).

beginner

How do you calculate the average of a column named sales after grouping by region?

Use df.groupBy('region').avg('sales') to get the average sales per region.

intermediate

What is the difference between agg() and direct aggregation functions like sum() after groupBy()?

agg() lets you apply multiple aggregation functions at once, while direct functions like sum() apply a single aggregation.

beginner

Why is grouping and aggregation useful in real life?

It helps summarize large data sets, like finding total sales per store or average temperature per city, making data easier to understand.

What does df.groupBy('category').count() do?

A Counts the number of columns in the DataFrame

B Counts the total number of categories

C Counts the number of unique values in the DataFrame

D Counts the number of rows in each category group

The count() after groupBy() counts rows per group.

Which function calculates the sum of a column after grouping?

A sum()

B avg()

C count()

D max()

sum() adds up values in each group.

How do you apply multiple aggregations like sum and average together?

A Use multiple groupBy() calls

B Use agg() with a dictionary of functions

C Use count() twice

D Use select() only

The agg() function allows multiple aggregations at once.

What type of object does groupBy() return?

A GroupedData

B DataFrame

C RDD

D List

groupBy() returns a GroupedData object to apply aggregations.

Which of these is NOT an aggregation function in Spark?

A max()

B min()

C filter()

D avg()

filter() is for filtering rows, not aggregation.

Explain how you would use groupBy() and aggregation functions to find the total sales per product category.

Think about grouping by category and then summing sales.

Describe the difference between using agg() and direct aggregation functions after groupBy().

Consider when you want one vs many summary stats.