CatBoost in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main advantage of CatBoost's handling of categorical features?

CatBoost is known for its special way of dealing with categorical data. What is the main advantage of this approach compared to traditional methods like one-hot encoding?

A It ignores categorical features to speed up training.

B It converts categories into numerical values using target statistics without data leakage.

C It converts categories into random numbers to increase model randomness.

D It requires manual one-hot encoding before training.

💡 Hint

Think about how CatBoost prevents information from leaking from the target variable into the feature transformation.

CatBoost uses a technique called ordered target statistics to convert categorical features into numbers. This method avoids data leakage by calculating statistics in a way that does not use the target value of the current row during training.

❓ Model Choice

intermediate

Choosing CatBoost for a dataset with many categorical features

You have a dataset with 50 categorical features and 10 numerical features. You want a model that handles categorical data well without extensive preprocessing. Which model is the best choice?

A Linear regression, because it is simple and fast.

B Support Vector Machine, because it works well with categorical data.

C K-Nearest Neighbors, because it handles categorical data well by default.

D CatBoost, because it natively supports categorical features.

💡 Hint

Consider which model can directly use categorical features without manual encoding.

CatBoost is designed to handle categorical features natively, which means it can use them directly without manual encoding. Other models usually require preprocessing steps like one-hot encoding.

❓ Metrics

advanced

Interpreting CatBoost training output metrics

During CatBoost training, you see the following output for a binary classification task:

Iteration 100: train Logloss = 0.25, validation Logloss = 0.30

What does this tell you about the model's performance?

A The model perfectly fits both training and validation data.

B The model is underfitting because both losses are low.

C The model fits training data well but may be overfitting since validation loss is higher.

D The model is not learning because validation loss is higher than training loss.

💡 Hint

Compare training and validation losses to understand model fit.

The training loss is lower than validation loss, which suggests the model fits training data well but may not generalize perfectly to unseen data, indicating possible overfitting.

🔧 Debug

advanced

Why does CatBoost training raise a 'CatBoostError: Categorical feature is not converted'?

You try to train a CatBoost model with categorical features but get this error:

CatBoostError: Categorical feature is not converted

What is the most likely cause?

A You did not specify which columns are categorical in the CatBoost Pool or model parameters.

B Your dataset contains missing values in numerical columns.

C You used a regression loss function for classification.

D You did not normalize the numerical features before training.

💡 Hint

Think about how CatBoost knows which features are categorical.

CatBoost requires you to explicitly tell it which features are categorical by passing their indices or names. If you don't, it treats them as numerical and raises this error when it encounters unexpected data types.

❓ Hyperparameter

expert

Optimizing CatBoost with the 'depth' hyperparameter

You want to improve your CatBoost model's performance on a complex dataset. You consider increasing the 'depth' parameter from 6 to 10. What is the most likely effect?

A The model may capture more complex patterns but risks overfitting and longer training time.

B The model will train faster and generalize better with higher depth.

C The model's performance will not change because depth only affects numerical features.

D The model will ignore categorical features with higher depth.

💡 Hint

Think about how tree depth affects model complexity and training time.

Increasing tree depth allows the model to learn more complex patterns but can lead to overfitting and slower training because trees become larger and more detailed.

Practice

1. What is the main advantage of using CatBoost in machine learning?

easy

A. It handles categorical features automatically without extensive preprocessing

B. It requires manual encoding of all categorical variables

C. It only works with numerical data

D. It is slower than most other boosting algorithms

Solution

1. Step 1: Understand CatBoost's feature handling

   CatBoost is designed to handle categorical features internally, so you don't need to manually encode them.

2. Step 2: Compare with other algorithms

   Other algorithms often require manual encoding like one-hot or label encoding, which CatBoost avoids.

3. Final Answer:

   It handles categorical features automatically without extensive preprocessing -> Option A

4. Quick Check:

   CatBoost = automatic categorical handling [OK]

Hint: Remember CatBoost means 'Categorical Boosting' [OK]

Common Mistakes:

• Thinking CatBoost needs manual encoding
• Assuming CatBoost only works with numbers
• Believing CatBoost is slower than others

2. Which of the following is the correct way to import CatBoostClassifier in Python?

easy

A. from catboost import classifier

B. from catboost import CatBoostClassifier

C. import CatBoost from catboost

D. import catboost.CatBoostClassifier

Solution

1. Step 1: Recall Python import syntax for CatBoost

   The correct import statement uses 'from catboost import CatBoostClassifier' to import the classifier class.

2. Step 2: Check other options for syntax errors

   Options A, B, and D have incorrect syntax or wrong class names.

3. Final Answer:

   from catboost import CatBoostClassifier -> Option B

4. Quick Check:

   Correct import = from catboost import CatBoostClassifier [OK]

Hint: Use 'from catboost import CatBoostClassifier' always [OK]

Common Mistakes:

• Using wrong import syntax
• Incorrect class name capitalization
• Trying to import with dot notation

3. What will be the output of the following code snippet?

from catboost import CatBoostClassifier
X = [[1, 'red'], [2, 'blue'], [3, 'green']]
y = [0, 1, 0]
model = CatBoostClassifier(iterations=10, verbose=False)
model.fit(X, y, cat_features=[1])
preds = model.predict([[2, 'red']])
print(preds.tolist())

medium

A. [2]

B. [1]

C. [0]

D. Error due to categorical feature

Solution

1. Step 1: Understand training data and labels

   The model is trained on 3 samples with categorical feature at index 1 and labels 0 or 1.

2. Step 2: Predict on new sample [2, 'red']

   The model predicts the class for this input. Since 'red' was seen with label 0, prediction is likely 0.

3. Final Answer:

   [0] -> Option C

4. Quick Check:

   Prediction matches label 0 for 'red' [OK]

Hint: Check training labels for matching category [OK]

Common Mistakes:

• Assuming prediction is 1 without checking labels
• Expecting error due to categorical feature
• Confusing feature index for cat_features

4. Identify the error in this CatBoost training code:

from catboost import CatBoostClassifier
X = [[1, 'red'], [2, 'blue'], [3, 'green']]
y = [0, 1, 0]
model = CatBoostClassifier(iterations=10)
model.fit(X, y)

medium

A. Missing cat_features parameter for categorical data

B. Incorrect label format

C. Wrong import statement

D. iterations parameter must be a string

Solution

1. Step 1: Check data and model parameters

   The data contains a categorical feature (strings) but cat_features is not specified.

2. Step 2: Understand CatBoost requirements

   CatBoost needs to know which features are categorical to handle them properly.

3. Final Answer:

   Missing cat_features parameter for categorical data -> Option A

4. Quick Check:

   cat_features required for categorical columns [OK]

Hint: Always specify cat_features for categorical columns [OK]

Common Mistakes:

• Forgetting cat_features causes poor model or error
• Assuming CatBoost auto-detects categories
• Misusing iterations parameter

5. You want to train a CatBoostClassifier on a dataset with 3 categorical features and 5 numerical features. Which approach is best to maximize model performance?

hard

A. Convert all categorical features to one-hot encoding before training

B. Use CatBoost without specifying cat_features and increase iterations to 1000

C. Ignore categorical features and train only on numerical features

D. Specify the indices of the 3 categorical features in cat_features and use default parameters

Solution

1. Step 1: Understand CatBoost's handling of categorical features

   CatBoost performs best when categorical features are specified via cat_features so it can handle them internally.

2. Step 2: Evaluate other options

   One-hot encoding is unnecessary and can increase dimensionality; ignoring categorical features loses information; not specifying cat_features prevents CatBoost from using its special handling.

3. Final Answer:

   Specify the indices of the 3 categorical features in cat_features and use default parameters -> Option D

4. Quick Check:

   Best practice = specify cat_features [OK]

Hint: Always tell CatBoost which features are categorical [OK]

Common Mistakes:

• One-hot encoding categorical features manually
• Ignoring categorical features
• Not specifying cat_features and expecting best results