Shift and lag operations in Data Analysis Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of shift operation on a pandas Series

What is the output of the following code snippet?

Data Analysis Python

import pandas as pd
s = pd.Series([10, 20, 30, 40, 50])
result = s.shift(2)
print(result)

A

0    10
1    20
2    30
3    40
4    50
dtype: int64

B

0     NaN
1     NaN
2    10.0
3    20.0
4    30.0
dtype: float64

C

0    30.0
1    40.0
2    50.0
3     NaN
4     NaN
dtype: float64

D

0    NaN
1    10.0
2    20.0
3    30.0
4    40.0
dtype: float64

💡 Hint

Shift moves values down by the number specified, filling new spots with NaN.

The shift(2) moves all values down by 2 places. The first two positions become NaN, and the rest are the original values shifted down.

❓ data_output

intermediate

Result of lag operation with fill value

Given the DataFrame below, what is the output of the lag operation with fill value 0?

Data Analysis Python

import pandas as pd
df = pd.DataFrame({'A': [5, 10, 15, 20]})
df['lag_A'] = df['A'].shift(1, fill_value=0)
print(df)

A

    A  lag_A
0   5      0
1  10      5
2  15     10
3  20     15

B

    A  lag_A
0   5    NaN
1  10      5
2  15     10
3  20     15

C

    A  lag_A
0   5      0
1  10      0
2  15      5
3  20     10

D

    A  lag_A
0   5      5
1  10     10
2  15     15
3  20     20

💡 Hint

The fill_value replaces NaN created by the shift.

The shift(1) moves values down by 1. The first row gets the fill_value 0 instead of NaN.

❓ visualization

advanced

Visualizing shift effect on time series data

Which plot correctly shows the original and shifted time series data?

Data Analysis Python

import pandas as pd
import matplotlib.pyplot as plt
s = pd.Series([3, 6, 9, 12, 15])
s_shifted = s.shift(1)
plt.plot(s, label='Original')
plt.plot(s_shifted, label='Shifted')
plt.legend()
plt.show()

A Line plot with original series starting at 3 and shifted series starting with NaN at index 0

B Bar plot with original and shifted series overlapping exactly

C Scatter plot with shifted series values all zero

D Line plot with shifted series values ahead of original by one index

💡 Hint

Shifted series moves values down, so first value is missing (NaN).

The line plot shows original values starting at index 0, shifted values start with NaN at index 0 and values from original shifted down by one index.

🔧 Debug

advanced

Identify error in shift usage

What error will this code raise and why? import pandas as pd s = pd.Series([1, 2, 3]) result = s.shift('2')

Data Analysis Python

import pandas as pd
s = pd.Series([1, 2, 3])
result = s.shift('2')

A KeyError because '2' is not a valid index

B ValueError because string '2' is not a valid shift value

C No error, shift converts string '2' to integer 2 automatically

D TypeError because shift expects an integer or timedelta, not a string

💡 Hint

Check the type of argument shift accepts.

shift expects an integer or timedelta. Passing a string causes a TypeError.

🧠 Conceptual

expert

Understanding difference between shift and lag in time series

Which statement best describes the difference between shift and lag operations in time series data?

A Shift and lag are identical operations with different names in pandas

B Shift moves data up or down by a specified number of periods, while lag always moves data backward by one period

C Shift moves data by a specified number of periods and can move forward or backward; lag specifically refers to moving data backward by one or more periods

D Lag moves data forward in time, shift moves data backward

💡 Hint

Think about direction and flexibility of shift vs lag.

Shift is a general operation that can move data forward or backward by any number of periods. Lag is a specific case of shift moving data backward (to past periods).

Practice

1. What does the shift(1) function do to a pandas DataFrame column?

easy

A. Moves the data down by one row, creating a lag of previous values

B. Moves the data up by one row, creating a lead of next values

C. Sorts the column in ascending order

D. Removes the first row of the column

Solution

1. Step 1: Understand the shift function

   The shift(1) moves data down by one row, so each value is replaced by the previous row's value.

2. Step 2: Identify lag vs lead

   Positive shift values create lag (previous rows), so shift(1) creates lag by one row.

3. Final Answer:

   Moves the data down by one row, creating a lag of previous values -> Option A

4. Quick Check:

   shift(1) = lag by 1 row [OK]

Hint: Positive shift means lag (previous rows), negative means lead [OK]

Common Mistakes:

• Confusing shift(1) with shift(-1)
• Thinking shift sorts data
• Assuming shift removes rows

2. Which of the following is the correct syntax to create a new column 'prev_value' that contains the previous row's value of column 'value' in pandas?

easy

A. df['prev_value'] = df['value'].shift(-1)

B. df['prev_value'] = shift(df['value'], 1)

C. df['prev_value'] = df.shift('value', 1)

D. df['prev_value'] = df['value'].shift(1)

Solution

1. Step 1: Recall pandas shift syntax

   The correct syntax is to call shift() on the Series, like df['value'].shift(1).

2. Step 2: Check each option

   df['prev_value'] = df['value'].shift(1) uses correct syntax. df['prev_value'] = df['value'].shift(-1) shifts in the wrong direction. df['prev_value'] = df.shift('value', 1) uses wrong method call. df['prev_value'] = shift(df['value'], 1) uses undefined function.

3. Final Answer:

   df['prev_value'] = df['value'].shift(1) -> Option D

4. Quick Check:

   Correct pandas shift syntax = df['prev_value'] = df['value'].shift(1) [OK]

Hint: Use Series.shift(n) to lag by n rows [OK]

Common Mistakes:

• Using negative shift for previous row
• Calling shift on DataFrame with wrong arguments
• Using undefined shift function

3. Given the DataFrame:

import pandas as pd
df = pd.DataFrame({'value': [10, 20, 30, 40]})
df['lag1'] = df['value'].shift(1)
df['lead1'] = df['value'].shift(-1)
print(df)

What is the output?

medium

A. value lag1 lead1
0 10 10.0 20.0
1 20 20.0 30.0
2 30 30.0 40.0
3 40 40.0 NaN

B. value lag1 lead1
0 10 20.0 NaN
1 20 30.0 10.0
2 30 40.0 20.0
3 40 NaN 30.0

C. value lag1 lead1
0 10 NaN 20.0
1 20 10.0 30.0
2 30 20.0 40.0
3 40 30.0 NaN

D. value lag1 lead1
0 10 NaN NaN
1 20 10.0 20.0
2 30 20.0 30.0
3 40 30.0 40.0

Solution

1. Step 1: Calculate lag1 column

   shift(1) moves values down by one, so first row is NaN, second row gets 10, third 20, fourth 30.

2. Step 2: Calculate lead1 column

   shift(-1) moves values up by one, so last row is NaN, third row gets 40, second 30, first 20.

3. Final Answer:

   value lag1 lead1 0 10 NaN 20.0 1 20 10.0 30.0 2 30 20.0 40.0 3 40 30.0 NaN -> Option C

4. Quick Check:

   Positive shift = lag, negative shift = lead [OK]

Hint: Positive shift adds NaN on top, negative shift adds NaN at bottom [OK]

Common Mistakes:

• Mixing lag and lead directions
• Forgetting NaN appears at shifted edges
• Confusing shift with sorting

4. What is wrong with this code snippet?

df['prev'] = df.shift(1)['value']

Assuming df is a pandas DataFrame with a column 'value'.

medium

A. It works correctly and creates a lagged column

B. It shifts the entire DataFrame, not just the 'value' column, which is inefficient

C. It causes a syntax error due to wrong indexing

D. It will cause a KeyError because shift returns a Series, not DataFrame

Solution

1. Step 1: Understand what df.shift(1) does

   df.shift(1) shifts the entire DataFrame down by one row, including all columns.

2. Step 2: Check indexing and efficiency

   Accessing ['value'] after shift works, but shifting entire DataFrame is inefficient if only one column is needed.

3. Final Answer:

   It shifts the entire DataFrame, not just the 'value' column, which is inefficient -> Option B

4. Quick Check:

   Shift on DataFrame shifts all columns [OK]

Hint: Shift only the needed column, not whole DataFrame [OK]

Common Mistakes:

• Assuming shift returns Series when called on DataFrame
• Expecting syntax error from indexing
• Ignoring performance impact

5. You have a time series DataFrame df with a column 'sales'. You want to create a new column 'pct_change' that shows the percentage change from the previous day. Which code correctly uses shift to calculate this?

hard

A. df['pct_change'] = (df['sales'] - df['sales'].shift(1)) / df['sales'].shift(1) * 100

B. df['pct_change'] = (df['sales'].shift(1) - df['sales']) / df['sales'] * 100

C. df['pct_change'] = df['sales'].pct_change(1)

D. df['pct_change'] = df['sales'].shift(-1) / df['sales'] * 100

Solution

1. Step 1: Understand percentage change formula

   Percentage change = (current - previous) / previous * 100.

2. Step 2: Apply shift to get previous value

   df['sales'].shift(1) gives previous day's sales, so formula in df['pct_change'] = (df['sales'] - df['sales'].shift(1)) / df['sales'].shift(1) * 100 matches the formula.

3. Step 3: Check other options

   df['pct_change'] = (df['sales'].shift(1) - df['sales']) / df['sales'] * 100 reverses numerator and denominator incorrectly. df['pct_change'] = df['sales'].pct_change(1) uses built-in method but question asks for shift usage. df['pct_change'] = df['sales'].shift(-1) / df['sales'] * 100 uses lead instead of lag.

4. Final Answer:

   df['pct_change'] = (df['sales'] - df['sales'].shift(1)) / df['sales'].shift(1) * 100 -> Option A

5. Quick Check:

   Percentage change uses lag with shift(1) [OK]

Hint: Use (current - shift(1)) / shift(1) for percent change [OK]

Common Mistakes:

• Using shift(-1) instead of shift(1)
• Reversing numerator and denominator
• Confusing built-in pct_change with manual shift