Forward fill and backward fill in Data Analysis Python - Step-by-Step Execution

Concept Flow - Forward fill and backward fill

Start with DataFrame with missing values

↓

Choose fill method: forward fill or backward fill

↓

Forward fill

↓

Fill missing with

↓

previous valid

↓

Return filled DataFrame

Start with data having missing values, choose forward or backward fill to replace missing values with previous or next valid data, then get the filled data.

Execution Sample

Data Analysis Python

import pandas as pd

df = pd.DataFrame({'A': [1, None, None, 4, None, 6]})

ffill_df = df.ffill()
bfill_df = df.bfill()

This code creates a DataFrame with missing values and fills them forward and backward.

Execution Table

Step	Index	Original Value	Forward Fill Value	Backward Fill Value	Explanation
1	0	1	1	1	No missing, value stays the same
2	1	NaN	1	4	Forward fill uses previous 1, backward fill uses next 4
3	2	NaN	1	4	Same as above, fills missing
4	3	4	4	4	No missing, value stays the same
5	4	NaN	4	6	Forward fill uses previous 4, backward fill uses next 6
6	5	6	6	6	No missing, value stays the same
7	-	-	-	-	End of data, all missing filled

💡 All missing values replaced by forward or backward fill, no NaNs remain.

Variable Tracker

Variable	Start	After ffill	After bfill
df['A']	[1, NaN, NaN, 4, NaN, 6]	[1, 1, 1, 4, 4, 6]	[1, 4, 4, 4, 6, 6]

Key Moments - 3 Insights

Why does forward fill replace missing values with the previous valid value?

Forward fill looks up the column and fills each missing spot with the last known valid value above it, as shown in execution_table rows 2 and 3.

What happens if the first value is missing when using forward fill?

Forward fill cannot fill missing values at the start because there is no previous valid value; those remain NaN unless backward fill or other methods are used.

How does backward fill differ from forward fill in filling missing values?

Backward fill fills missing values using the next valid value below, as shown in execution_table rows 2 and 3 where missing values are replaced by 4 instead of 1.

Visual Quiz - 3 Questions

Look at the execution table, what is the forward fill value at index 4?

A NaN

B 4

C 6

D 1

💡 Hint

Check row 5 in the execution_table under Forward Fill Value.

At which index does the backward fill replace a missing value with 4?

A Index 1

B Index 5

C Index 0

D Index 4

💡 Hint

Look at execution_table rows 2 and 3 under Backward Fill Value.

If the first value in the column was NaN, what would forward fill do to it?

A Fill it with zero

B Fill it with the next valid value

C Leave it as NaN

D Fill it with the last valid value

💡 Hint

Forward fill uses previous valid values; no previous value means no fill (see key_moments).

Concept Snapshot

Forward fill (ffill) replaces missing values with the last known valid value above.
Backward fill (bfill) replaces missing values with the next valid value below.
Use df.ffill() or df.bfill() in pandas.
Missing values at start (ffill) or end (bfill) may remain if no valid neighbor.
Useful for filling gaps in time series or ordered data.

Full Transcript

We start with a DataFrame that has missing values. Forward fill replaces each missing value with the previous valid value found above it in the column. Backward fill replaces missing values with the next valid value found below. For example, at index 1 and 2, forward fill uses the value 1 from index 0, while backward fill uses the value 4 from index 3. Missing values at the start cannot be forward filled because there is no previous value. This method helps fill gaps in data smoothly. The execution table shows each step's original and filled values. The variable tracker shows how the column changes after each fill method. Understanding these fills helps handle missing data in real datasets.

Practice

1. What does forward fill do when handling missing data in a dataset?

easy

A. It removes rows with missing values.

B. It copies the next known value backward to fill missing spots.

C. It replaces missing values with zero.

D. It copies the last known value forward to fill missing spots.

Solution

1. Step 1: Understand forward fill concept

   Forward fill means filling missing data by copying the last known value forward to the next missing spot.

2. Step 2: Compare options with definition

   "It copies the last known value forward to fill missing spots." matches this definition exactly, while others describe different methods.

3. Final Answer:

   It copies the last known value forward to fill missing spots. -> Option D

4. Quick Check:

   Forward fill = copy last known value forward [OK]

Hint: Forward fill copies previous value forward to fill gaps [OK]

Common Mistakes:

• Confusing forward fill with backward fill
• Thinking forward fill removes missing data
• Assuming forward fill replaces with zero

2. Which of the following is the correct syntax to apply forward fill on a pandas DataFrame df?

easy

A. df.fillna(method='ffill')

B. df.fillna(method='bfill')

C. df.fillna(forward=True)

D. df.fillna(direction='forward')

Solution

1. Step 1: Recall pandas fillna syntax

   In pandas, forward fill is done by fillna(method='ffill').

2. Step 2: Check options for correct syntax

   df.fillna(method='ffill') matches the correct method name 'ffill'. df.fillna(method='bfill') performs backward fill. The remaining options use invalid syntax.

3. Final Answer:

   df.fillna(method='ffill') -> Option A

4. Quick Check:

   Forward fill method = 'ffill' [OK]

Hint: Use method='ffill' for forward fill in pandas [OK]

Common Mistakes:

• Using 'bfill' instead of 'ffill' for forward fill
• Passing invalid parameters like forward=True
• Confusing method names in fillna

3. Given the DataFrame df below, what will be the result of df.bfill()?

import pandas as pd
df = pd.DataFrame({'A': [1, None, None, 4, None]})
print(df.bfill())

medium

A. A
0 1.0
1 4.0
2 4.0
3 4.0
4 NaN

B. A
0 1.0
1 1.0
2 1.0
3 4.0
4 NaN

C. A
0 1.0
1 NaN
2 NaN
3 4.0
4 NaN

D. A
0 1.0
1 4.0
2 4.0
3 4.0
4 4.0

Solution

1. Step 1: Understand backward fill on given data

   Backward fill copies the next known value backward to fill missing spots. Here, missing values at index 1 and 2 get filled with 4.0 from index 3. The last missing at index 4 has no next value, so remains NaN.

2. Step 2: Match output with options

   A 0 1.0 1 4.0 2 4.0 3 4.0 4 NaN matches this result exactly. A 0 1.0 1 4.0 2 4.0 3 4.0 4 4.0 incorrectly fills last NaN with 4.0, others do not match.

3. Final Answer:

   A 0 1.0 1 4.0 2 4.0 3 4.0 4 NaN -> Option A

4. Quick Check:

   Backward fill fills missing with next known value [OK]

Hint: Backward fill uses next known value to fill missing [OK]

Common Mistakes:

• Assuming last NaN gets filled when no next value exists
• Confusing forward and backward fill results
• Misreading index positions in output

4. You run the code below but get an error. What is the mistake?

import pandas as pd
df = pd.DataFrame({'A': [None, 2, None]})
df.fillna(method='forward')

medium

A. fillna requires inplace=True to work.

B. DataFrame cannot have None values.

C. The method name 'forward' is invalid; should be 'ffill'.

D. fillna does not accept method parameter.

Solution

1. Step 1: Check fillna method parameter

   The valid method names are 'ffill' for forward fill and 'bfill' for backward fill. 'forward' is not valid.

2. Step 2: Identify cause of error

   Using 'forward' causes a ValueError. DataFrames can have None values, fillna does not require inplace=True, and it accepts the method parameter.

3. Final Answer:

   The method name 'forward' is invalid; should be 'ffill'. -> Option C

4. Quick Check:

   Use 'ffill' for forward fill method [OK]

Hint: Use 'ffill' not 'forward' as method name in fillna [OK]

Common Mistakes:

• Using 'forward' instead of 'ffill'
• Thinking inplace=True is mandatory
• Believing fillna can't handle None values

5. You have a DataFrame df with missing values scattered in column 'B'. You want to fill missing values by first forward filling, then backward filling any remaining missing values. Which code correctly does this?

hard

A. df['B'].fillna(method='ffill', method='bfill')

B. df['B'].fillna(method='ffill').fillna(method='bfill')

C. df['B'].fillna(method='bfill').fillna(method='ffill')

D. df['B'].fillna(method=['ffill', 'bfill'])

Solution

1. Step 1: Understand chaining fillna calls

   To fill missing values first forward then backward, chain fillna calls: first with method='ffill', then on result with method='bfill'.

2. Step 2: Evaluate options for correct chaining

   df['B'].fillna(method='ffill').fillna(method='bfill') correctly chains fillna twice. df['B'].fillna(method='bfill').fillna(method='ffill') reverses order. Attempting multiple methods in one call (duplicate kwargs or list) uses invalid syntax.

3. Final Answer:

   df['B'].fillna(method='ffill').fillna(method='bfill') -> Option B

4. Quick Check:

   Chain fillna with 'ffill' then 'bfill' [OK]

Hint: Chain fillna with 'ffill' then 'bfill' to fill all gaps [OK]

Common Mistakes:

• Trying to pass multiple methods in one fillna call
• Reversing forward and backward fill order
• Not chaining calls, so some NaNs remain