shift() for lagging data in Pandas - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the shift() function do in pandas?

The shift() function moves the data in a column up or down by a specified number of rows, creating lagged or lead data. It is often used to compare current values with past or future values.

beginner

How do you create a new column with the previous day's value using shift()?

Use df['previous_day'] = df['value'].shift(1). This moves the 'value' column down by one row, so each row shows the value from the previous day.

beginner

What happens to the first row when you use shift(1) on a column?

The first row becomes NaN because there is no previous row to fill it with. This is normal when creating lagged data.

intermediate

Can shift() be used to lead data instead of lag data? How?

Yes. Use a negative number like shift(-1) to move data up, creating lead data (future values).

beginner

Why is lagging data useful in data science?

Lagging data helps compare current values with past values. It is useful for time series analysis, trend detection, and building features for models that depend on past information.

What does df['value'].shift(2) do?

A Moves the 'value' column down by 2 rows, creating lagged data.

B Moves the 'value' column up by 2 rows, creating lead data.

C Deletes 2 rows from the 'value' column.

D Duplicates the 'value' column twice.

shift(2) moves data down by 2 rows, so each row shows the value from 2 rows above.

What value appears in the first row after applying shift(1)?

A NaN (missing value)

B The original first row value

C Zero

D The last row value

The first row has no previous row to fill from, so it becomes NaN.

How do you create a column with next day's data using shift()?

A <code>shift(1)</code>

B <code>shift()</code> without arguments

C <code>shift(-1)</code>

D <code>shift(0)</code>

Using shift(-1) moves data up by one row, showing next day's data.

Which of these is a common use of lagged data?

A Changing data types

B Comparing current sales with past sales

C Sorting data alphabetically

D Removing duplicates

Lagged data helps compare current values with past values, useful in time series.

If you want to compare today's temperature with the temperature 3 days ago, which code is correct?

A df['temp_3days_ago'] = df['temperature']

B df['temp_3days_ago'] = df['temperature'].shift(-3)

C df['temp_3days_ago'] = df['temperature'].shift(1)

D df['temp_3days_ago'] = df['temperature'].shift(3)

Shift by 3 moves data down 3 rows, showing temperature from 3 days ago.

Explain how the shift() function helps create lagged data in pandas and why this is useful.

Think about how you compare today's data with yesterday's.

Describe what happens to the first few rows of a DataFrame column when you apply shift(1) and how you might handle those values.

Consider what data is missing after shifting.