Bird
Raised Fist0

Handling Missing Data - Imputation Strategies and When to Drop

THEORYintermediateInterview freq: high
Choose your preparation mode6 modes available
▶
Execution Trace
Step 1/4
Missing Data?

Dataset contains 15% missing values in 'Age' column.

🎯 Identify columns with significant missing data.
missing_data = df.isnull().sum()
missing_percentage = missing_data / len(df) * 100
Imputation Strategy

Mean imputation applied to 'Age' column.

🎯 Choose imputation based on data distribution.
df['Age'].fillna(df['Age'].mean(), inplace=True)
Imputation Strategy

KNN imputation used for 'Income' column.

🎯 Use advanced imputation for complex patterns.
from sklearn.impute import KNNImputer
imputer = KNNImputer(n_neighbors=5)
df[['Income']] = imputer.fit_transform(df[['Income']])
Drop Missing Data?

Dropped 'Address' column due to 60% missing data.

🎯 Drop columns with excessive missing data.
df.drop(columns=['Address'], inplace=True)
📊
Diagram
Diagram
┌───────────────────────────────┐
│       Missing Data?           │
└────────────┬──────────────────┘
             │
             ▼
┌───────────────────────────────┐
│   Imputation Strategy         │
│   1. Mean/Median/Mode         │
│   2. KNN Imputation           │
│   3. Predictive Models        │
└────────────┬──────────────────┘
             │
             ▼
┌───────────────────────────────┐
│   Drop Missing Data?          │
│   - If too much missing       │
│   - If not critical feature   │
└───────────────────────────────┘
  • Missing Data? - Initial check for missing values
  • Imputation Strategy - Methods to fill missing values
  • Drop Missing Data? - Criteria for dropping data
🔑
Key Moments
▲
Q: Why might mean imputation be inappropriate for skewed data?
A: Mean imputation can distort the distribution of skewed data, leading to biased results.
Q: When is it better to drop data rather than impute?
A: Dropping data is preferable when the missing data is excessive or the feature is not critical.
Q: How does KNN imputation differ from mean imputation?
A: KNN imputation uses neighboring data points to estimate missing values, capturing more complex patterns.

The diagram explains how to handle missing data in datasets. It starts by identifying missing values, then deciding between imputation strategies such as mean, median, mode, KNN, or predictive models. If the missing data is excessive or the feature is not critical, dropping the data might be the best option. The trace steps illustrate practical applications of these strategies, highlighting when to use each method. Key moments test understanding of why certain strategies are chosen over others, ensuring a deep grasp of handling missing data effectively.

Primary Focus

✓ Handling Missing Data

Practice

(1/5)
1. Given the following Python code snippet for a segmentation task, what will be the output shape if the input image shape is (256, 256, 3) and the model outputs a segmentation map with 10 classes?
import numpy as np
input_image = np.random.rand(256, 256, 3)
model_output = np.random.rand(256, 256, 10)
easy
A. (256, 256, 10)
B. (256, 256, 1)
C. (256, 256, 3)
D. (256, 256)

Solution

  1. Step 1: Understand the model output

    The model outputs a segmentation map with a shape corresponding to the input spatial dimensions and the number of classes.
  2. Step 2: Eliminate wrong options

    (256, 256, 3) is the input shape, (256, 256, 1) implies binary segmentation, and (256, 256) lacks class dimension.
  3. Final Answer:

    Option A -> Option A
  4. Quick Check:

    Output shape = Input spatial + class channels [OK]
Hint: Output shape = Input spatial + class channels [OK]
Common Mistakes:
2. Given the following Python code using Naive Bayes, what will be the output?
from sklearn.naive_bayes import GaussianNB
import numpy as np
X = np.array([[1, 2], [2, 3], [3, 4]])
y = np.array([1, 0, 1])
model = GaussianNB()
model.fit(X, y)
print(model.predict([[2, 2]]))
easy
A. [1]
B. [0]
C. [0, 1]
D. Error due to shape mismatch

Solution

  1. Step 1: Understand the Code Execution

    The code fits a Gaussian Naive Bayes model and predicts the class for a new sample [2, 2].
  2. Step 2: Analyze the Prediction

    Given the training data, the model predicts class 1 for [2, 2] based on the likelihoods.
  3. Final Answer:

    Option A -> Option A
  4. Quick Check:

    GaussianNB predicts based on likelihoods. [[check]]
Hint: GaussianNB predicts based on likelihoods. [[check]]
Common Mistakes:
3. In the context of evaluating models on imbalanced datasets, which metric is more sensitive to changes in the minority class predictions?
easy
A. ROC-AUC
B. Accuracy
C. PR-AUC
D. F1-Score

Solution

  1. Step 1: Understand metric sensitivity

    PR-AUC focuses on precision and recall, which are directly affected by the minority class predictions.
  2. Step 2: Eliminate wrong options

    ROC-AUC is less sensitive to class imbalance. Accuracy can be misleading in imbalanced datasets. F1-Score is related but not as directly sensitive as PR-AUC.
  3. Final Answer:

    Option C -> Option C
  4. Quick Check:

    PR-AUC focuses on minority class. [OK]
Hint: PR-AUC focuses on minority class. [OK]
Common Mistakes:
4.
import numpy as np
from keras.models import Sequential
from keras.layers import Dense

model = Sequential()
model.add(Dense(1, input_dim=2, activation='relu'))
model.add(Dense(1, activation='relu'))

weights = model.get_weights()
print([w.shape for w in weights])
What does this code output?
easy
A. [(2, 1), (1,), (1, 1), (1,)]
B. [(2, 1), (1,), (1, 2), (1,)]
C. [(1, 2), (1,), (1, 1), (1,)]
D. [(2, 1), (2,), (1, 1), (1,)]

Solution

  1. Step 1: Analyze the network structure

    The first layer has 2 inputs and 1 output, so weights are (2, 1) and biases are (1,). The second layer has 1 input and 1 output, so weights are (1, 1) and biases are (1,).
  2. Step 2: Eliminate wrong options

    [(2, 1), (1,), (1, 2), (1,)] is incorrect because the second layer's weights should be (1, 1). [(1, 2), (1,), (1, 1), (1,)] is incorrect because the first layer's weights should be (2, 1). [(2, 1), (2,), (1, 1), (1,)] is incorrect because the first layer's biases should be (1,).
  3. Final Answer:

    Option A -> Option A
  4. Quick Check:

    Check layer dimensions match input/output. [OK]
Hint: Check layer dimensions match input/output. [OK]
Common Mistakes:
5. How would you adapt K-Means to handle a dataset with categorical features?
hard
A. Use one-hot encoding and apply standard K-Means
B. Use K-Modes clustering instead
C. Convert categorical features to numerical using label encoding
D. Ignore categorical features and cluster on numerical features only

Solution

  1. Step 1: Understand K-Means limitations

    K-Means is not suitable for categorical data as it relies on Euclidean distance.
  2. Step 2: Choose the right adaptation

    K-Modes is designed for clustering categorical data, making it the best choice.
  3. Final Answer:

    Option B -> Option B
  4. Quick Check:

    K-Modes for categorical data [OK]
Hint: K-Modes for categorical data [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard