Target encoding in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is target encoding in machine learning?

Target encoding is a technique that replaces categorical values with the average of the target variable for those categories. It helps models use categorical data effectively.

intermediate

Why do we use target encoding instead of one-hot encoding for high-cardinality features?

Target encoding reduces the number of new features created, avoiding a large increase in data size and helping models learn better from categories with many unique values.

intermediate

How does target encoding help prevent overfitting?

By using techniques like smoothing and cross-validation, target encoding avoids leaking target information from the training set to the model, which helps prevent overfitting.

advanced

What is smoothing in target encoding?

Smoothing blends the category's target mean with the overall target mean to reduce noise from categories with few samples, making the encoding more stable.

beginner

Give a simple example of target encoding for a categorical feature with three categories and their target means.

If a feature has categories A, B, C with target means 0.2, 0.5, and 0.8 respectively, target encoding replaces A with 0.2, B with 0.5, and C with 0.8 in the dataset.

What does target encoding replace a categorical value with?

A A unique integer ID

B A one-hot vector

C The average target value for that category

D The frequency of the category

Target encoding replaces each category with the average of the target variable for that category.

Why is smoothing used in target encoding?

A To reduce noise from categories with few samples

B To increase the number of features

C To speed up encoding

D To convert numerical data to categorical

Smoothing helps reduce noise by blending category target means with the overall mean, especially for rare categories.

Which problem can target encoding help solve better than one-hot encoding?

A Handling missing values

B Encoding high-cardinality categorical features

C Scaling numerical features

D Reducing dataset size by removing features

Target encoding is useful for high-cardinality categorical features where one-hot encoding would create too many new columns.

What is a risk of using target encoding without precautions?

A Model underfitting

B Loss of categorical information

C Slower training time

D Data leakage leading to overfitting

Without careful handling, target encoding can leak target information into features, causing overfitting.

Which method helps avoid overfitting when applying target encoding?

A Using cross-validation to compute encodings

B Ignoring rare categories

C Using one-hot encoding instead

D Normalizing the target variable

Computing target encodings using cross-validation prevents the model from seeing the target values of the same data points during training.

Explain what target encoding is and why it is useful for categorical data.

Think about how categorical values can be turned into numbers that carry information about the target.

Describe how smoothing and cross-validation help make target encoding more reliable.

Consider how to avoid mistakes when using target information to encode features.