Decision tree classifier in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

How does a decision tree classifier decide the best split?

Imagine you want to split your data at a node in a decision tree. What criterion does the tree use to choose the best feature and value to split on?

A It splits to create the largest possible groups regardless of class labels.

B It randomly picks any feature and value without considering the data.

C It chooses the split that maximizes the information gain or reduces impurity the most.

D It always splits on the feature with the highest numeric value.

💡 Hint

Think about how the tree tries to make groups that are more pure or similar.

A decision tree chooses splits that maximize information gain or reduce impurity (like Gini impurity or entropy), making child nodes more homogeneous.

❓ Predict Output

intermediate

Output of training a decision tree on simple data

What is the accuracy printed by this code after training a decision tree on the given data?

ML Python

from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score

X = [[0], [1], [2], [3]]
y = [0, 0, 1, 1]

model = DecisionTreeClassifier(random_state=42)
model.fit(X, y)
pred = model.predict(X)
acc = accuracy_score(y, pred)
print(acc)

A 1.0

B 0.5

C 0.75

D 0.0

💡 Hint

The model is trained and tested on the same data.

The decision tree perfectly fits the training data, so accuracy is 1.0.

❓ Hyperparameter

advanced

Effect of max_depth on decision tree complexity

What happens if you set the max_depth parameter of a decision tree classifier to a very small number like 1?

A The tree will randomly choose splits without considering max_depth.

B The tree will grow very deep and overfit the data.

C The tree will ignore the max_depth and grow fully.

D The tree will be very simple, possibly underfitting the data.

💡 Hint

Think about how limiting depth affects the tree's ability to separate classes.

Setting max_depth to 1 limits the tree to only one split, which may not capture complex patterns, causing underfitting.

❓ Metrics

advanced

Choosing the right metric for imbalanced classes

You trained a decision tree classifier on a dataset where 95% of samples belong to class A and 5% to class B. Which metric is best to evaluate your model's performance?

A Precision and recall, because they focus on minority class performance.

B Accuracy, because it shows overall correct predictions.

C Mean squared error, because it measures prediction error.

D Confusion matrix size, because bigger matrices mean better models.

💡 Hint

Think about why accuracy might be misleading with imbalanced data.

Accuracy can be high by just predicting the majority class. Precision and recall help measure how well the model detects the minority class.

🔧 Debug

expert

Why does this decision tree code raise an error?

What error does this code raise and why?

from sklearn.tree import DecisionTreeClassifier

X = [[1, 2], [3, 4]]
y = [0, 1, 0]

model = DecisionTreeClassifier()
model.fit(X, y)

A TypeError: DecisionTreeClassifier() got an unexpected keyword argument.

B ValueError: Found input variables with inconsistent numbers of samples because X has 2 samples but y has 3 labels.

C IndexError: list index out of range during fitting.

D No error, the model fits successfully.

💡 Hint

Check if the number of samples in X matches the number of labels in y.

The feature matrix X has 2 samples but the label vector y has 3 labels, causing a mismatch error.

Practice

1. What is the main purpose of a Decision Tree Classifier in machine learning?

easy

A. To store data in a database

B. To perform mathematical calculations on numbers

C. To generate random numbers for simulations

D. To split data into groups based on simple questions for classification

Solution

1. Step 1: Understand the role of decision trees

   Decision tree classifiers split data by asking simple yes/no questions to separate data into classes.

2. Step 2: Compare options with the purpose

   Only To split data into groups based on simple questions for classification describes splitting data for classification, which is the main use of decision trees.

3. Final Answer:

   To split data into groups based on simple questions for classification -> Option D

4. Quick Check:

   Decision tree purpose = classification by splitting [OK]

Hint: Decision trees split data step-by-step for classification [OK]

Common Mistakes:

• Confusing decision trees with random number generators
• Thinking decision trees store data like databases
• Assuming decision trees perform complex math calculations

2. Which of the following is the correct way to create a Decision Tree Classifier in Python using scikit-learn?

easy

A. from sklearn.tree import DecisionTreeClassifier
model = DecisionTreeClassifier()

B. from sklearn.tree import DecisionTree
model = DecisionTree()

C. import DecisionTreeClassifier from sklearn
model = DecisionTreeClassifier()

D. from sklearn.classifier import DecisionTreeClassifier
model = DecisionTreeClassifier()

Solution

1. Step 1: Recall correct import syntax

   The correct import for decision tree classifier is from sklearn.tree import DecisionTreeClassifier.

2. Step 2: Check object creation

   Creating the model with DecisionTreeClassifier() is correct. Other options have wrong imports or class names.

3. Final Answer:

   from sklearn.tree import DecisionTreeClassifier model = DecisionTreeClassifier() -> Option A

4. Quick Check:

   Correct import and class name = from sklearn.tree import DecisionTreeClassifier model = DecisionTreeClassifier() [OK]

Hint: Use sklearn.tree and DecisionTreeClassifier() to create model [OK]

Common Mistakes:

• Using wrong import paths like sklearn.classifier
• Misspelling class name as DecisionTree
• Incorrect import syntax like import DecisionTreeClassifier from sklearn

3. Given the following code, what will be the predicted class for the input X_test = [[5, 3]]?

from sklearn.tree import DecisionTreeClassifier
X_train = [[2, 3], [10, 15], [6, 8], [1, 1]]
y_train = [0, 1, 1, 0]
model = DecisionTreeClassifier(max_depth=2)
model.fit(X_train, y_train)
print(model.predict(X_test))

medium

A. [0]

B. [1]

C. [2]

D. Error: max_depth must be greater than 2

Solution

1. Step 1: Understand training data and labels

   Training data has points labeled 0 or 1. Points near [5,3] are closer to class 1 points like [6,8].

2. Step 2: Predict with trained model

   The decision tree with max_depth=2 will classify [5,3] as class 1 based on training data proximity.

3. Final Answer:

   [1] -> Option B

4. Quick Check:

   Prediction for [5,3] = [1] [OK]

Hint: Check nearest training points to predict class [OK]

Common Mistakes:

• Assuming max_depth=2 causes error
• Predicting class 2 which does not exist
• Ignoring training labels when predicting

4. What is wrong with this code snippet for training a Decision Tree Classifier?

from sklearn.tree import DecisionTreeClassifier
model = DecisionTreeClassifier(max_depth=-1)
model.fit(X_train, y_train)

medium

A. X_train and y_train are not defined

B. DecisionTreeClassifier does not have max_depth parameter

C. max_depth cannot be negative; it should be null or positive

D. fit() method requires extra parameters

Solution

1. Step 1: Check max_depth parameter validity

   max_depth must be null or a positive integer. Negative values are invalid and cause errors.

2. Step 2: Verify other code parts

   Assuming X_train and y_train are defined, and fit() requires only these two parameters, the main error is max_depth.

3. Final Answer:

   max_depth cannot be negative; it should be null or positive -> Option C

4. Quick Check:

   max_depth negative = error [OK]

Hint: max_depth must be positive or null, never negative [OK]

Common Mistakes:

• Thinking max_depth can be negative
• Assuming fit() needs more parameters
• Confusing parameter names

5. You want to avoid overfitting in a Decision Tree Classifier trained on a noisy dataset. Which approach is best to control this?

hard

A. Limit the tree depth using the max_depth parameter

B. Increase the number of features used for splitting

C. Train the tree without any depth limit

D. Use a decision tree without pruning

Solution

1. Step 1: Understand overfitting in decision trees

   Overfitting happens when the tree is too deep and fits noise instead of general patterns.

2. Step 2: Choose method to reduce overfitting

   Limiting max_depth controls tree size and prevents overfitting. Increasing features or no pruning can worsen it.

3. Final Answer:

   Limit the tree depth using the max_depth parameter -> Option A

4. Quick Check:

   Control overfitting = limit max_depth [OK]

Hint: Limit tree depth to prevent overfitting [OK]

Common Mistakes:

• Thinking more features reduce overfitting
• Believing no depth limit is better
• Ignoring pruning or depth control