SVM for text classification in NLP

Introduction

SVM helps us sort text into groups by finding the best line that separates different categories clearly.

Sorting emails into spam or not spam

Classifying movie reviews as positive or negative

Organizing news articles by topic

Filtering customer feedback into categories

Detecting language of short text messages

Syntax

NLP

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.svm import SVC

# Convert text to numbers
vectorizer = TfidfVectorizer()
X_train = vectorizer.fit_transform(train_texts)
X_test = vectorizer.transform(test_texts)

# Create SVM model
model = SVC(kernel='linear')

# Train model
model.fit(X_train, train_labels)

# Predict new data
predictions = model.predict(X_test)

TfidfVectorizer changes text into numbers that the SVM can understand.

Using a 'linear' kernel is common for text because it works well with many features.

Examples

This sets up a simple SVM that draws a straight line to separate classes.

NLP

model = SVC(kernel='linear')

This removes common English words like 'the' or 'and' to focus on important words.

NLP

vectorizer = TfidfVectorizer(stop_words='english')

This line asks the model to guess the categories for new text data.

NLP

predictions = model.predict(X_test)

Sample Model

This program trains an SVM to tell positive and negative movie reviews apart using simple example sentences. It then predicts new sentences and shows how accurate it is.

NLP

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score

# Sample text data
train_texts = [
    'I love this movie',
    'This film was terrible',
    'Amazing story and great acting',
    'Worst movie ever',
    'I enjoyed the film a lot'
]
train_labels = [1, 0, 1, 0, 1]  # 1=positive, 0=negative

test_texts = [
    'I hate this movie',
    'What a fantastic film'
]

def main():
    # Convert text to numbers
    vectorizer = TfidfVectorizer(stop_words='english')
    X_train = vectorizer.fit_transform(train_texts)
    X_test = vectorizer.transform(test_texts)

    # Create and train SVM model
    model = SVC(kernel='linear')
    model.fit(X_train, train_labels)

    # Predict test data
    predictions = model.predict(X_test)

    # Show predictions
    print('Predictions:', predictions.tolist())

    # For demonstration, assume true labels for test
    true_labels = [0, 1]
    accuracy = accuracy_score(true_labels, predictions)
    print(f'Accuracy: {accuracy:.2f}')

if __name__ == '__main__':
    main()

Important Notes

SVM works well with many features, which is common in text data after vectorization.

Choosing the right text vectorizer (like TF-IDF) helps the SVM focus on important words.

Linear kernel is usually enough for text classification, making training faster.

Summary

SVM finds the best line to separate text categories.

Text must be changed into numbers before using SVM.

TF-IDF vectorizer and linear kernel are common choices for text classification.

Practice

1. What is the main purpose of using an SVM (Support Vector Machine) in text classification?

easy

A. To find the best line that separates different text categories

B. To count the number of words in the text

C. To translate text into another language

D. To generate random text samples

Solution

1. Step 1: Understand SVM's role in classification

   SVM tries to find a boundary (line or hyperplane) that best separates different classes in data.

2. Step 2: Apply this to text classification

   In text classification, SVM finds the best line to separate categories like spam vs. not spam.

3. Final Answer:

   To find the best line that separates different text categories -> Option A

4. Quick Check:

   SVM separates classes = D [OK]

Hint: SVM separates classes by finding the best boundary line [OK]

Common Mistakes:

• Thinking SVM counts words directly
• Confusing SVM with translation tools
• Assuming SVM generates text

2. Which of the following is the correct way to convert text data before applying an SVM model in Python?

easy

A. Use CountVectorizer() or TfidfVectorizer() to transform text into numbers

B. Directly feed raw text strings into the SVM model

C. Use OneHotEncoder() on raw text strings

D. Apply StandardScaler() on raw text strings

Solution

1. Step 1: Identify text preprocessing for SVM

   SVM requires numeric input, so text must be converted to numbers using vectorizers like CountVectorizer or TfidfVectorizer.

2. Step 2: Check other options

   Raw text cannot be fed directly; OneHotEncoder and StandardScaler are not suitable for raw text strings.

3. Final Answer:

   Use CountVectorizer() or TfidfVectorizer() to transform text into numbers -> Option A

4. Quick Check:

   Text to numbers = Vectorizer = C [OK]

Hint: Always vectorize text before SVM, never raw strings [OK]

Common Mistakes:

• Feeding raw text directly to SVM
• Using OneHotEncoder on text strings
• Applying scalers on text without vectorizing

3. Given the following Python code snippet, what will be the output of print(predicted_labels)?

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.svm import LinearSVC

texts = ["I love cats", "Dogs are great", "Cats are cute", "I hate dogs"]
labels = [1, 0, 1, 0]

vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(texts)

model = LinearSVC()
model.fit(X, labels)

new_texts = ["I love dogs", "Cats are great"]
X_new = vectorizer.transform(new_texts)
predicted_labels = model.predict(X_new)

medium

A. [1, 0]

B. [0, 1]

C. [1, 1]

D. [0, 0]

Solution

1. Step 1: Understand training labels and texts

   Texts labeled 1 are about cats, 0 about dogs. Model learns cats=1, dogs=0.

2. Step 2: Predict new texts

   "I love dogs" likely labeled 0 (dog), "Cats are great" labeled 1 (cat).

3. Final Answer:

   [0, 1] -> Option B

4. Quick Check:

   Dog text=0, Cat text=1 = B [OK]

Hint: Match new text topics to training labels for quick guess [OK]

Common Mistakes:

• Mixing label meanings
• Assuming model predicts opposite labels
• Ignoring vectorizer effect

4. You trained an SVM model for text classification but got an error: ValueError: could not convert string to float. What is the most likely cause?

medium

A. You set the wrong kernel parameter in SVM

B. You used too many training samples

C. You forgot to convert text data into numeric vectors before training

D. You used a linear kernel instead of RBF kernel

Solution

1. Step 1: Analyze the error message

   The error means the model received raw text strings instead of numbers.

2. Step 2: Identify cause in text classification

   Text must be vectorized (converted to numbers) before training SVM.

3. Final Answer:

   You forgot to convert text data into numeric vectors before training -> Option C

4. Quick Check:

   Raw text input causes conversion error = A [OK]

Hint: Check if text is vectorized before training SVM [OK]

Common Mistakes:

• Ignoring need for vectorization
• Blaming kernel choice for conversion errors
• Assuming data size causes this error

5. You want to improve your SVM text classifier's performance on a dataset with many common words like "the", "and", "is". Which approach is best to try?

hard

A. Switch to a polynomial kernel without changing text preprocessing

B. Increase the SVM regularization parameter without changing vectorization

C. Use raw word counts without removing stop words

D. Use a TF-IDF vectorizer to reduce the impact of common words

Solution

1. Step 1: Understand the problem with common words

   Common words appear everywhere and do not help distinguish classes well.

2. Step 2: Choose vectorization method to reduce common word impact

   TF-IDF lowers weights of common words, improving model focus on important words.

3. Step 3: Evaluate other options

   Changing regularization or kernel without addressing common words won't help much.

4. Final Answer:

   Use a TF-IDF vectorizer to reduce the impact of common words -> Option D

5. Quick Check:

   TF-IDF reduces common word weight = A [OK]

Hint: TF-IDF downweights common words, improving text classification [OK]

Common Mistakes:

• Ignoring stop words effect
• Changing SVM parameters without vectorizing
• Using raw counts with many common words