diff() for differences in Pandas - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the diff() function do in pandas?

The diff() function calculates the difference between consecutive rows or elements in a pandas Series or DataFrame.

beginner

How do you calculate the difference between rows in a DataFrame using diff()?

Call diff() on the DataFrame. By default, it subtracts the previous row from the current row for each column.

intermediate

What does the periods parameter in diff() control?

The periods parameter sets how many rows (or elements) to go back when calculating the difference. For example, periods=2 subtracts the value two rows above.

beginner

What happens if the first row has no previous row when using diff()?

The first row will have a NaN value because there is no previous row to subtract from.

beginner

Can diff() be used on a pandas Series? What is the result?

Yes, diff() works on a Series and returns a new Series with the difference between each element and the previous one.

What does df.diff() compute by default?

A Sum of all rows

B Cumulative sum of rows

C Difference between columns

D Difference between each row and the previous row

diff() calculates the difference between each row and the previous row by default.

What value does diff() return for the first row?

A NaN

B 1

C 0

D Same as first row

The first row has no previous row, so diff() returns NaN.

How do you calculate the difference between a row and the row two places above using diff()?

A <code>df.diff(axis=1)</code>

B <code>df.diff(periods=1)</code>

C <code>df.diff(periods=2)</code>

D <code>df.diff(periods=-2)</code>

Use periods=2 to subtract the value two rows above.

Can diff() be applied to a pandas Series?

A No, only DataFrames

B Yes, it returns differences between elements

C Yes, but returns the sum

D No, it causes an error

diff() works on Series and returns the difference between consecutive elements.

What does df.diff(axis=1) do?

A Calculates difference between columns for each row

B Calculates difference between rows for each column

C Calculates cumulative sum

D Returns the original DataFrame

Setting axis=1 calculates difference between columns horizontally for each row.

Explain how the diff() function works in pandas and give an example of when you might use it.

Think about how you find change from one day to the next in a list of numbers.

Describe the role of the periods parameter in diff() and how changing it affects the output.

Imagine looking back more than one step to find differences.

Practice

1. What does the diff() function in pandas do when applied to a DataFrame column?

easy

A. Calculates the difference between each value and the previous value in the column

B. Calculates the sum of all values in the column

C. Sorts the column values in ascending order

D. Replaces missing values with zeros

Solution

1. Step 1: Understand the purpose of diff()

   The diff() function calculates the difference between each element and the one before it in the same column.

2. Step 2: Compare options with the function's behavior

   Only Calculates the difference between each value and the previous value in the column correctly describes this behavior. Other options describe different functions.

3. Final Answer:

   Calculates the difference between each value and the previous value in the column -> Option A

4. Quick Check:

   diff() = difference between consecutive values [OK]

Hint: Remember, diff() shows step-by-step changes in data [OK]

Common Mistakes:

• Thinking diff() sums values
• Confusing diff() with sorting
• Assuming diff() fills missing data

2. Which of the following is the correct syntax to calculate the difference between rows in a pandas DataFrame column named sales?

easy

A. df.diff('sales')

B. df['sales'].diff()

C. df.diff[sales]()

D. df['sales'].difference()

Solution

1. Step 1: Identify correct method call on a DataFrame column

   To calculate differences on a column, call diff() on the Series: df['sales'].diff().

2. Step 2: Check syntax of each option

   df['sales'].diff() uses correct syntax. df.diff('sales') wrongly passes column name to diff. df.diff[sales]() uses wrong brackets. df['sales'].difference() uses a non-existent method.

3. Final Answer:

   df['sales'].diff() -> Option B

4. Quick Check:

   Correct syntax = df['column'].diff() [OK]

Hint: Use diff() directly on the column Series like df['col'].diff() [OK]

Common Mistakes:

• Passing column name as argument to diff()
• Using square brackets instead of parentheses
• Calling non-existent methods like difference()

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'A': [10, 15, 20, 18]})
df['diff'] = df['A'].diff()

What is the content of the diff column?

medium

A. [NaN, 10, 15, 20]

B. [10, 15, 20, 18]

C. [NaN, 5.0, 5.0, -2.0]

D. [0, 5, 5, -2]

Solution

1. Step 1: Calculate differences between consecutive values in column 'A'

   Values: 10, 15, 20, 18. Differences: NaN (first row), 15-10=5, 20-15=5, 18-20=-2.

2. Step 2: Match calculated differences with options

   [NaN, 5.0, 5.0, -2.0] matches the list [NaN, 5.0, 5.0, -2.0]. Others do not.

3. Final Answer:

   [NaN, 5.0, 5.0, -2.0] -> Option C

4. Quick Check:

   diff() = [NaN, 5, 5, -2] [OK]

Hint: First diff is always NaN; subtract previous value from current [OK]

Common Mistakes:

• Assuming first diff is zero instead of NaN
• Confusing original values with differences
• Ignoring negative differences

4. You run this code:

import pandas as pd
df = pd.DataFrame({'X': [3, 7, 2]})
df['change'] = df.diff(axis=1)

But you get an error or unexpected output. What is the problem?

medium

A. diff() cannot be used with axis=1 on a single-column DataFrame

B. You must specify the period argument in diff()

C. The DataFrame must have numeric column names for diff()

D. diff() requires the DataFrame to have at least 5 rows

Solution

1. Step 1: Understand axis parameter in diff()

   axis=1 means calculate difference across columns (left to right). With only one column, no previous column exists to subtract.

2. Step 2: Identify why error or unexpected output occurs

   Since only one column exists, diff(axis=1) returns NaN or error because no previous column to compare.

3. Final Answer:

   diff() cannot be used with axis=1 on a single-column DataFrame -> Option A

4. Quick Check:

   diff(axis=1) needs multiple columns [OK]

Hint: diff(axis=1) needs 2+ columns; single column causes NaN/error [OK]

Common Mistakes:

• Thinking period argument is mandatory
• Assuming column names must be numeric
• Believing diff() needs many rows

5. You have a DataFrame showing daily temperatures:

import pandas as pd
data = {'Day': ['Mon', 'Tue', 'Wed', 'Thu'], 'Temp': [20, 22, 21, 25]}
df = pd.DataFrame(data)

You want to add a column Temp_change showing how much temperature changed from the previous day. Which code correctly does this?

hard

A. df['Temp_change'] = df['Temp'].cumsum()

B. df['Temp_change'] = df['Temp'].diff(periods=2)

C. df['Temp_change'] = df['Temp'].shift()

D. df['Temp_change'] = df['Temp'].diff()

Solution

1. Step 1: Identify how to calculate daily temperature change

   Temperature change from previous day is difference between current and previous value, which diff() calculates by default with period=1.

2. Step 2: Evaluate each option

   df['Temp_change'] = df['Temp'].diff() uses diff() correctly. df['Temp_change'] = df['Temp'].cumsum() sums temperatures, not differences. df['Temp_change'] = df['Temp'].shift() shifts values but does not calculate difference. df['Temp_change'] = df['Temp'].diff(periods=2) calculates difference over 2 days, not 1.

3. Final Answer:

   df['Temp_change'] = df['Temp'].diff() -> Option D

4. Quick Check:

   Use diff() for stepwise changes [OK]

Hint: Use diff() without arguments for previous row difference [OK]

Common Mistakes:

• Using cumsum() instead of diff()
• Using shift() which only moves data
• Using diff(periods=2) for daily change