Support Vector Machine (SVM) in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main goal of a Support Vector Machine (SVM)?

The main goal of an SVM is to find the best boundary (called a hyperplane) that separates different groups (classes) in the data with the largest possible margin.

beginner

What is a 'margin' in the context of SVM?

The margin is the distance between the hyperplane and the closest data points from each class. SVM tries to maximize this margin to improve the model's ability to generalize.

beginner

What are 'support vectors' in SVM?

Support vectors are the data points closest to the hyperplane. They are critical because they define the position and orientation of the hyperplane.

intermediate

How does SVM handle data that is not linearly separable?

SVM uses a technique called the 'kernel trick' to transform data into a higher dimension where it can be separated by a hyperplane, even if it is not separable in the original space.

intermediate

What is the role of the 'C' parameter in SVM?

The 'C' parameter controls the trade-off between having a wide margin and correctly classifying training points. A small C allows more misclassifications but a wider margin, while a large C tries to classify all points correctly but may have a smaller margin.

What does SVM try to maximize when finding the best hyperplane?

A The training data size

B The number of support vectors

C The number of features

D The margin between classes

SVM aims to maximize the margin, which is the distance between the hyperplane and the nearest data points of each class.

Which points influence the position of the SVM hyperplane?

A Random points

B All data points

C Only the support vectors

D Points farthest from the hyperplane

Only the support vectors, the closest points to the hyperplane, determine its position.

What technique does SVM use to handle non-linear data?

A Kernel trick

B Gradient descent

C Decision trees

D Random forests

The kernel trick transforms data into a higher dimension to make it linearly separable.

What happens if the 'C' parameter in SVM is set very high?

A The model allows more misclassifications

B The model tries to classify all training points correctly

C The margin becomes wider

D The model ignores support vectors

A high 'C' value forces the model to classify all training points correctly, possibly at the cost of a smaller margin.

Which of the following is NOT true about support vectors?

A They are always misclassified points

B They define the margin width

C They are the closest points to the hyperplane

D They influence the hyperplane position

Support vectors are not necessarily misclassified; they are the closest points to the hyperplane from each class.

Explain in your own words how SVM finds the best boundary to separate two groups.

Think about the line or plane that best divides the groups with the biggest gap.

Describe how the kernel trick helps SVM deal with data that is not linearly separable.

Imagine bending or stretching the space so groups can be separated by a straight line.

Practice

1. What is the main goal of a Support Vector Machine (SVM) in machine learning?

easy

A. To cluster data points into groups without labels

B. To reduce the number of features in the dataset

C. To find the best boundary that separates different classes

D. To generate random predictions for data points

Solution

1. Step 1: Understand SVM's purpose

   SVM aims to separate data points into classes by drawing a boundary.

2. Step 2: Identify the best boundary

   SVM finds the boundary (called hyperplane) that maximizes the margin between classes.

3. Final Answer:

   To find the best boundary that separates different classes -> Option C

4. Quick Check:

   SVM goal = best separating boundary [OK]

Hint: SVM separates classes with the widest margin [OK]

Common Mistakes:

• Confusing SVM with clustering methods
• Thinking SVM reduces features
• Believing SVM makes random predictions

2. Which of the following is the correct way to import the SVM classifier from scikit-learn in Python?

easy

A. import svm from sklearn

B. from sklearn.svm import SVC

C. from sklearn import svm.SVC

D. import SVC from sklearn.svm

Solution

1. Step 1: Recall scikit-learn import syntax

   Correct import uses 'from module import class' format.

2. Step 2: Match correct syntax for SVM

   The SVM classifier is called SVC and is imported from sklearn.svm as 'from sklearn.svm import SVC'.

3. Final Answer:

   from sklearn.svm import SVC -> Option B

4. Quick Check:

   Correct import = from sklearn.svm import SVC [OK]

Hint: Use 'from sklearn.svm import SVC' to import SVM [OK]

Common Mistakes:

• Using wrong import order
• Trying to import class directly without module
• Using invalid syntax like 'import svm from sklearn'

3. What will be the output of the following Python code using SVM?

from sklearn.svm import SVC
import numpy as np

X = np.array([[0, 0], [1, 1]])
y = [0, 1]
model = SVC(kernel='linear')
model.fit(X, y)
pred = model.predict([[2, 2]])
print(pred)

medium

A. [0]

B. Error

C. [2]

D. [1]

Solution

1. Step 1: Understand training data and labels

   Training points: [0,0] labeled 0 and [1,1] labeled 1.

2. Step 2: Predict label for [2,2]

   [2,2] is closer to class 1 side since it lies beyond [1,1]. So prediction is 1.

3. Final Answer:

   [1] -> Option D

4. Quick Check:

   Point [2,2] predicted class = 1 [OK]

Hint: Points near class 1 get label 1 [OK]

Common Mistakes:

• Predicting label 0 for point far from class 0
• Expecting error due to small data
• Confusing kernel effect on prediction

4. Identify the error in this SVM code snippet:

from sklearn.svm import SVC
X = [[1, 2], [2, 3], [3, 4]]
y = [0, 1]
model = SVC()
model.fit(X, y)

medium

A. Mismatch in number of samples and labels

B. Wrong import statement

C. Kernel parameter missing

D. Predict method not called

Solution

1. Step 1: Check data and label lengths

   X has 3 samples but y has only 2 labels, causing mismatch.

2. Step 2: Understand fit requirements

   fit() requires equal number of samples and labels to train properly.

3. Final Answer:

   Mismatch in number of samples and labels -> Option A

4. Quick Check:

   Samples count != labels count causes error [OK]

Hint: Samples and labels must match in length [OK]

Common Mistakes:

• Ignoring length mismatch
• Thinking kernel is mandatory
• Expecting error without calling predict

5. You have a dataset with two classes that are not linearly separable. Which SVM kernel should you choose to improve classification?

hard

A. RBF (Radial Basis Function) kernel

B. Polynomial kernel

C. Linear kernel

D. Sigmoid kernel

Solution

1. Step 1: Understand data separability

   Non-linearly separable data means a straight line can't separate classes well.

2. Step 2: Choose kernel for complex boundaries

   RBF kernel maps data into higher dimension to separate complex shapes effectively.

3. Final Answer:

   RBF (Radial Basis Function) kernel -> Option A

4. Quick Check:

   Non-linear data needs RBF kernel [OK]

Hint: Use RBF kernel for non-linear data [OK]

Common Mistakes:

• Choosing linear kernel for non-linear data
• Confusing polynomial and RBF kernels
• Ignoring kernel choice impact