Interpolation for missing values in Pandas - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of linear interpolation on a DataFrame column

What is the output of the following code snippet that uses linear interpolation to fill missing values in a pandas DataFrame column?

Pandas

import pandas as pd
import numpy as np
df = pd.DataFrame({'A': [1, np.nan, np.nan, 4, 5]})
df['A'] = df['A'].interpolate(method='linear')
print(df['A'].tolist())

A [1.0, nan, nan, 4.0, 5.0]

B [1.0, 2.0, 3.0, 4.0, 5.0]

C [1.0, 2.5, 3.5, 4.0, 5.0]

D [1.0, 1.0, 1.0, 4.0, 5.0]

💡 Hint

Linear interpolation fills missing values by connecting points with a straight line.

The missing values at positions 1 and 2 are filled by linear interpolation between 1 and 4, resulting in 2.0 and 3.0 respectively.

❓ data_output

intermediate

Result of time interpolation on a time-indexed DataFrame

Given a DataFrame indexed by dates with missing values, what is the result of using time-based interpolation?

Pandas

import pandas as pd
import numpy as np
idx = pd.to_datetime(['2024-01-01', '2024-01-02', '2024-01-04', '2024-01-05'])
df = pd.DataFrame({'Value': [10, np.nan, 40, 50]}, index=idx)
df_interpolated = df.interpolate(method='time')
print(df_interpolated['Value'].tolist())

A [10.0, 25.0, 40.0, 50.0]

B [10.0, 30.0, 40.0, 50.0]

C [10.0, nan, 40.0, 50.0]

D [10.0, 20.0, 40.0, 50.0]

💡 Hint

Time interpolation considers the time difference between index points.

The missing value on 2024-01-02 is interpolated based on time distance between 2024-01-01 (10) and 2024-01-04 (40), resulting in 20.0 (1 day out of 3 days: 10 + 30 * (1/3)).

🔧 Debug

advanced

Identify the error in interpolation code

What error will this code raise when trying to interpolate missing values in a DataFrame?

Pandas

import pandas as pd
import numpy as np
df = pd.DataFrame({'A': [1, np.nan, 3]})
df['A'] = df['A'].interpolate(method='polynomial', order=2)
print(df)

A No error, outputs interpolated DataFrame

B TypeError: interpolate() got an unexpected keyword argument 'order'

C ValueError: method 'polynomial' requires specifying 'order' parameter

D AttributeError: 'Series' object has no attribute 'interpolate'

💡 Hint

Check if the 'order' parameter is correctly passed for polynomial interpolation.

The 'polynomial' method requires the 'order' parameter to be specified, otherwise it raises a ValueError.

🚀 Application

advanced

Choosing interpolation method for categorical data

You have a DataFrame column with missing categorical values like ['red', NaN, 'blue', NaN, 'green']. Which interpolation method is appropriate to fill missing values?

A Use method='polynomial' with order=2 for smooth category interpolation

B Use method='linear' to interpolate categories as numeric values

C Use method='nearest' to fill missing values with the closest non-missing category

D Use method='time' assuming the index is datetime

💡 Hint

Categorical data cannot be interpolated numerically.

Nearest method fills missing categorical values by copying the nearest valid category, which is suitable for categorical data.

🧠 Conceptual

expert

Effect of limit parameter in interpolation

What is the effect of setting the 'limit' parameter to 1 in pandas interpolate method when filling missing values?

A It limits the maximum number of consecutive NaNs to fill to 1, leaving longer gaps unfilled

B It limits the total number of NaNs filled in the entire DataFrame to 1

C It limits the interpolation to only the first missing value in the column

D It limits the interpolation to only numeric columns

💡 Hint

Think about how 'limit' controls consecutive missing values filled.

The 'limit' parameter restricts how many consecutive NaNs are filled. Setting limit=1 fills only one NaN in a row, leaving longer sequences partially unfilled.

Practice

1. What does the interpolate() method in pandas do when applied to a DataFrame with missing values?

easy

A. It fills missing values by estimating them based on nearby known values.

B. It deletes all rows containing missing values.

C. It replaces missing values with zeros.

D. It converts missing values to strings.

Solution

1. Step 1: Understand the purpose of interpolation

   Interpolation estimates missing data points by using values around them to create a smooth transition.

2. Step 2: Recognize what pandas interpolate() does

   The method fills missing values by calculating estimates based on existing nearby values, not by deleting or replacing with fixed values.

3. Final Answer:

   It fills missing values by estimating them based on nearby known values. -> Option A

4. Quick Check:

   Interpolation = estimate missing values [OK]

Hint: Interpolation estimates missing data using nearby values [OK]

Common Mistakes:

• Thinking it deletes rows with missing data
• Assuming it fills with zeros automatically
• Confusing interpolation with data type conversion

2. Which of the following is the correct syntax to interpolate missing values in a pandas DataFrame named df using the default method?

easy

A. df.interpolate(method='fill')

B. df.interpolate()

C. df.fillna(method='interpolate')

D. df.fillna(interpolate=True)

Solution

1. Step 1: Recall pandas interpolation syntax

   The method to fill missing values by interpolation is called directly as interpolate() on the DataFrame.

2. Step 2: Check each option for correctness

   df.interpolate() uses df.interpolate() which is correct. Options A, B, and D misuse fillna() or wrong parameters.

3. Final Answer:

   df.interpolate() -> Option B

4. Quick Check:

   Correct method call = df.interpolate() [OK]

Hint: Use df.interpolate() to fill missing values by default [OK]

Common Mistakes:

• Using fillna with wrong parameters
• Confusing interpolate with fillna method
• Adding invalid arguments to interpolate

3. Given the DataFrame df below, what will be the output of df.interpolate()?

import pandas as pd
import numpy as np
df = pd.DataFrame({'A': [1, np.nan, np.nan, 4, 5]})
print(df.interpolate())

medium

A. A
0 1.0
1 NaN
2 NaN
3 4.0
4 5.0

B. A
0 1.0
1 1.0
2 1.0
3 4.0
4 5.0

C. A
0 1.0
1 2.0
2 3.0
3 4.0
4 5.0

D. A
0 1.0
1 2.5
2 3.5
3 4.0
4 5.0

Solution

1. Step 1: Understand linear interpolation on missing values

   Linear interpolation fills missing values by evenly spacing values between known points. Here, missing values between 1 and 4 are filled as 2.0 and 3.0.

2. Step 2: Apply interpolation to the DataFrame

   Positions 1 and 2 are NaN. Interpolated values are 2.0 and 3.0 respectively, resulting in the sequence 1.0, 2.0, 3.0, 4.0, 5.0.

3. Final Answer:

   A 0 1.0 1 2.0 2 3.0 3 4.0 4 5.0 -> Option C

4. Quick Check:

   Linear interpolation fills gaps evenly [OK]

Hint: Linear interpolate fills gaps evenly between known values [OK]

Common Mistakes:

• Assuming missing values remain NaN
• Filling missing with previous value only
• Using wrong interpolation method

4. You run the code below but get an error. What is the most likely cause?

import pandas as pd
df = pd.DataFrame({'date': ['2023-01-01', '2023-01-02', '2023-01-03'], 'value': [1, None, 3]})
df['date'] = pd.to_datetime(df['date'])
df.interpolate(method='time')

medium

A. The DataFrame index is not set to the datetime column.

B. The 'value' column contains None instead of np.nan.

C. The 'date' column is not converted to datetime.

D. The interpolate method 'time' is not supported.

Solution

1. Step 1: Understand 'time' interpolation requirement

   The 'time' method requires the DataFrame index to be a datetime type to interpolate based on time differences.

2. Step 2: Check DataFrame index and columns

   Here, 'date' is a column but not set as index. Without datetime index, 'time' interpolation raises an error.

3. Final Answer:

   The DataFrame index is not set to the datetime column. -> Option A

4. Quick Check:

   Time interpolation needs datetime index [OK]

Hint: Set datetime column as index before time interpolation [OK]

Common Mistakes:

• Not setting datetime column as index
• Confusing None with np.nan (both work here)
• Assuming 'time' method unsupported

5. You have a DataFrame df with missing values in multiple columns and a datetime index. You want to fill missing values using time-based interpolation only on column 'temperature' without affecting others. Which code snippet correctly does this?

hard

A. df['temperature'].fillna(method='time')

B. df.interpolate(method='time', columns=['temperature'])

C. df.interpolate(method='time').loc[:, 'temperature']

D. df['temperature'] = df['temperature'].interpolate(method='time')

Solution

1. Step 1: Understand column-specific interpolation

   To interpolate only one column, call interpolate() on that column's Series.

2. Step 2: Check each option's correctness

   df['temperature'] = df['temperature'].interpolate(method='time') applies interpolation with method='time' directly on 'temperature' column and assigns back, which is correct. df.interpolate(method='time', columns=['temperature']) uses invalid 'columns' parameter. df.interpolate(method='time').loc[:, 'temperature'] interpolates whole DataFrame then selects column but does not assign back. df['temperature'].fillna(method='time') uses invalid fillna method.

3. Final Answer:

   df['temperature'] = df['temperature'].interpolate(method='time') -> Option D

4. Quick Check:

   Interpolate single column by calling method on that column [OK]

Hint: Interpolate single column by calling interpolate() on it [OK]

Common Mistakes:

• Using invalid 'columns' parameter in interpolate
• Not assigning interpolated result back to column
• Using fillna with method='time' which is invalid