diff() for differences in Pandas - Step-by-Step Execution

Concept Flow - diff() for differences

Start with DataFrame or Series

↓

Call diff() method

↓

Calculate difference between current and previous row

↓

Return new DataFrame/Series with differences

↓

Use or display result

The diff() method calculates the difference between each element and the previous one in a DataFrame or Series, returning a new object showing these differences.

Execution Sample

Pandas

import pandas as pd
s = pd.Series([10, 15, 20, 18])
diff_s = s.diff()
print(diff_s)

This code creates a Series and uses diff() to find the difference between each value and the one before it.

Execution Table

Step	Index	Current Value	Previous Value	Difference (Current - Previous)
1	0	10	None	NaN (no previous value)
2	1	15	10	5
3	2	20	15	5
4	3	18	20	-2
5	End	No more rows		Stop calculation

💡 Reached end of Series, no more previous values to subtract.

Variable Tracker

Variable	Start	After 1	After 2	After 3	Final
s	[10, 15, 20, 18]	[10, 15, 20, 18]	[10, 15, 20, 18]	[10, 15, 20, 18]	[10, 15, 20, 18]
diff_s	[NaN, NaN, NaN, NaN]	[NaN, 5, NaN, NaN]	[NaN, 5, 5, NaN]	[NaN, 5, 5, -2]	[NaN, 5, 5, -2]

Key Moments - 3 Insights

Why is the first difference value NaN?

The first value has no previous value to subtract from, so diff() returns NaN as shown in execution_table step 1.

What happens if the Series has missing values?

diff() subtracts the immediately previous value by position; if the previous value is NaN, the difference will be NaN.

Can diff() be used on DataFrames?

Yes, diff() works on DataFrames by default calculating difference row-wise for each column.

Visual Quiz - 3 Questions

Look at the execution_table at step 3, what is the difference calculated?

A 5

B 10

C -5

D NaN

💡 Hint

Check the 'Difference' column at step 3 in the execution_table.

At which step does the difference become negative?

A Step 2

B Step 4

C Step 3

D Never

💡 Hint

Look at the 'Difference' values in the execution_table rows.

If the first value was 0 instead of 10, what would be the difference at step 2?

A 15

B 10

C 5

D NaN

💡 Hint

Difference is current minus previous; changing first value affects subsequent differences.

Concept Snapshot

diff() method in pandas:
- Calculates difference between current and previous row values
- Returns NaN for first element (no previous)
- Works on Series and DataFrames
- Default axis=0 (row-wise differences)
- Useful for time series and sequential data analysis

Full Transcript

The diff() method in pandas calculates the difference between each element and the one before it in a Series or DataFrame. The first element has no previous value, so its difference is NaN. This method helps find changes or trends in data over time or sequence. For example, given a Series of numbers, diff() subtracts each number from the one before it and returns a new Series with these differences. This is useful in many data science tasks like analyzing stock price changes or sensor readings.