diff() for differences in Pandas - Mini Project: Build & Apply

Calculate Differences Using diff() in pandas

📖 Scenario: You work in a small shop that tracks daily sales numbers. You want to see how sales change from one day to the next.

🎯 Goal: Build a small program that uses pandas to find the daily difference in sales numbers using the diff() method.

📋 What You'll Learn

Create a pandas DataFrame with exact sales data for 5 days

Add a variable for the number of periods to calculate difference

Use the diff() method on the sales column with the periods variable

Print the DataFrame showing the original sales and the differences

💡 Why This Matters

🌍 Real World

Businesses often track changes in sales or other metrics day by day to understand trends and make decisions.

💼 Career

Data analysts and scientists use difference calculations to analyze time series data and detect patterns or anomalies.

1

Create the sales DataFrame

Create a pandas DataFrame called sales_data with a column 'Sales' containing these exact values: [100, 120, 115, 130, 125]. Use the index as days from 1 to 5.

Pandas

import pandas as pd
# Create the sales_data DataFrame with 'Sales' column and days 1 to 5 as index
# Your code here

Hint

Use pd.DataFrame with a dictionary for the column and set the index to days 1 through 5.

2

Set the periods variable

Create a variable called periods and set it to 1 to represent the difference between consecutive days.

Pandas

import pandas as pd
sales_data = pd.DataFrame({'Sales': [100, 120, 115, 130, 125]}, index=[1, 2, 3, 4, 5])
# Create the periods variable and set it to 1
# Your code here

Hint

Just create a variable named periods and assign it the value 1.

3

Calculate the daily sales difference

Create a new column in sales_data called 'Difference' by applying the diff() method on the 'Sales' column using the periods variable.

Pandas

import pandas as pd
sales_data = pd.DataFrame({'Sales': [100, 120, 115, 130, 125]}, index=[1, 2, 3, 4, 5])
periods = 1
# Add a 'Difference' column using diff() on 'Sales' with periods
# Your code here

Hint

Use sales_data['Sales'].diff(periods=periods) and assign it to sales_data['Difference'].

4

Print the sales data with differences

Print the sales_data DataFrame to show the original sales and the calculated daily differences.

Pandas

import pandas as pd
sales_data = pd.DataFrame({'Sales': [100, 120, 115, 130, 125]}, index=[1, 2, 3, 4, 5])
periods = 1
sales_data['Difference'] = sales_data['Sales'].diff(periods=periods)
# Print the sales_data DataFrame
# Your code here

Hint

Use print(sales_data) to show the DataFrame with the new column.

Practice

1. What does the diff() function in pandas do when applied to a DataFrame column?

easy

A. Calculates the difference between each value and the previous value in the column

B. Calculates the sum of all values in the column

C. Sorts the column values in ascending order

D. Replaces missing values with zeros

Solution

1. Step 1: Understand the purpose of diff()

   The diff() function calculates the difference between each element and the one before it in the same column.

2. Step 2: Compare options with the function's behavior

   Only Calculates the difference between each value and the previous value in the column correctly describes this behavior. Other options describe different functions.

3. Final Answer:

   Calculates the difference between each value and the previous value in the column -> Option A

4. Quick Check:

   diff() = difference between consecutive values [OK]

Hint: Remember, diff() shows step-by-step changes in data [OK]

Common Mistakes:

• Thinking diff() sums values
• Confusing diff() with sorting
• Assuming diff() fills missing data

2. Which of the following is the correct syntax to calculate the difference between rows in a pandas DataFrame column named sales?

easy

A. df.diff('sales')

B. df['sales'].diff()

C. df.diff[sales]()

D. df['sales'].difference()

Solution

1. Step 1: Identify correct method call on a DataFrame column

   To calculate differences on a column, call diff() on the Series: df['sales'].diff().

2. Step 2: Check syntax of each option

   df['sales'].diff() uses correct syntax. df.diff('sales') wrongly passes column name to diff. df.diff[sales]() uses wrong brackets. df['sales'].difference() uses a non-existent method.

3. Final Answer:

   df['sales'].diff() -> Option B

4. Quick Check:

   Correct syntax = df['column'].diff() [OK]

Hint: Use diff() directly on the column Series like df['col'].diff() [OK]

Common Mistakes:

• Passing column name as argument to diff()
• Using square brackets instead of parentheses
• Calling non-existent methods like difference()

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'A': [10, 15, 20, 18]})
df['diff'] = df['A'].diff()

What is the content of the diff column?

medium

A. [NaN, 10, 15, 20]

B. [10, 15, 20, 18]

C. [NaN, 5.0, 5.0, -2.0]

D. [0, 5, 5, -2]

Solution

1. Step 1: Calculate differences between consecutive values in column 'A'

   Values: 10, 15, 20, 18. Differences: NaN (first row), 15-10=5, 20-15=5, 18-20=-2.

2. Step 2: Match calculated differences with options

   [NaN, 5.0, 5.0, -2.0] matches the list [NaN, 5.0, 5.0, -2.0]. Others do not.

3. Final Answer:

   [NaN, 5.0, 5.0, -2.0] -> Option C

4. Quick Check:

   diff() = [NaN, 5, 5, -2] [OK]

Hint: First diff is always NaN; subtract previous value from current [OK]

Common Mistakes:

• Assuming first diff is zero instead of NaN
• Confusing original values with differences
• Ignoring negative differences

4. You run this code:

import pandas as pd
df = pd.DataFrame({'X': [3, 7, 2]})
df['change'] = df.diff(axis=1)

But you get an error or unexpected output. What is the problem?

medium

A. diff() cannot be used with axis=1 on a single-column DataFrame

B. You must specify the period argument in diff()

C. The DataFrame must have numeric column names for diff()

D. diff() requires the DataFrame to have at least 5 rows

Solution

1. Step 1: Understand axis parameter in diff()

   axis=1 means calculate difference across columns (left to right). With only one column, no previous column exists to subtract.

2. Step 2: Identify why error or unexpected output occurs

   Since only one column exists, diff(axis=1) returns NaN or error because no previous column to compare.

3. Final Answer:

   diff() cannot be used with axis=1 on a single-column DataFrame -> Option A

4. Quick Check:

   diff(axis=1) needs multiple columns [OK]

Hint: diff(axis=1) needs 2+ columns; single column causes NaN/error [OK]

Common Mistakes:

• Thinking period argument is mandatory
• Assuming column names must be numeric
• Believing diff() needs many rows

5. You have a DataFrame showing daily temperatures:

import pandas as pd
data = {'Day': ['Mon', 'Tue', 'Wed', 'Thu'], 'Temp': [20, 22, 21, 25]}
df = pd.DataFrame(data)

You want to add a column Temp_change showing how much temperature changed from the previous day. Which code correctly does this?

hard

A. df['Temp_change'] = df['Temp'].cumsum()

B. df['Temp_change'] = df['Temp'].diff(periods=2)

C. df['Temp_change'] = df['Temp'].shift()

D. df['Temp_change'] = df['Temp'].diff()

Solution

1. Step 1: Identify how to calculate daily temperature change

   Temperature change from previous day is difference between current and previous value, which diff() calculates by default with period=1.

2. Step 2: Evaluate each option

   df['Temp_change'] = df['Temp'].diff() uses diff() correctly. df['Temp_change'] = df['Temp'].cumsum() sums temperatures, not differences. df['Temp_change'] = df['Temp'].shift() shifts values but does not calculate difference. df['Temp_change'] = df['Temp'].diff(periods=2) calculates difference over 2 days, not 1.

3. Final Answer:

   df['Temp_change'] = df['Temp'].diff() -> Option D

4. Quick Check:

   Use diff() for stepwise changes [OK]

Hint: Use diff() without arguments for previous row difference [OK]

Common Mistakes:

• Using cumsum() instead of diff()
• Using shift() which only moves data
• Using diff(periods=2) for daily change