Shift and lag operations in Data Analysis Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does the shift() function do in data analysis?

The shift() function moves data up or down by a specified number of periods, creating a lag or lead effect. It helps compare current values with past or future values.

beginner

How is shift() different from lag()?

In pandas, shift() is used to create lag or lead values by shifting data. lag() is a similar concept but is more common in SQL or other tools. Both help compare values across time.

beginner

What happens to the data when you shift it forward by 1 period?

When you shift data forward by 1 period, each value moves down by one row, and the first row becomes empty (usually filled with NaN). This shows the previous value for each row.

intermediate

Why are shift and lag operations useful in time series analysis?

They help compare current data points with past or future points. This is useful to find trends, calculate differences, or create features like moving averages.

intermediate

How do you handle missing values created by shift operations?

Missing values appear because shifting moves data out of the original index. You can fill them using methods like fillna(), drop them, or keep them if they represent missing time points.

What does df['value'].shift(1) do to the 'value' column in a DataFrame?

A Duplicates the column

B Moves all values up by one row, inserting NaN at the bottom

C Sorts the column in ascending order

D Moves all values down by one row, inserting NaN at the top

Shifting by 1 moves values down by one row, so the first row becomes NaN.

If you want to compare today's sales with yesterday's sales, which operation would you use?

A shift(-1)

B sort()

C shift(1)

D groupby()

shift(1) moves data down, so you can compare current row with previous day's data.

What value typically fills the empty spots after a shift operation?

A 0

B NaN

C Empty string

D Original value

Empty spots after shifting are filled with NaN to indicate missing data.

Which pandas function is commonly used to fill missing values after a shift?

A fillna()

B dropna()

C sort_values()

D groupby()

fillna() replaces missing values with specified values or methods.

What does shift(-1) do?

A Moves data up by one row

B Duplicates data

C Moves data down by one row

D Deletes data

Negative shift moves data up, so the last row becomes NaN.

Explain how shift operations help in comparing values over time in a dataset.

Think about how you look at yesterday's weather to understand today's changes.

Describe a real-life example where you would use lag or shift operations in data analysis.

Consider how businesses track daily sales compared to previous days.