Text classification model in TensorFlow

Introduction

Text classification helps computers understand and sort text into groups. It makes tasks like spam detection or topic labeling easy.

Sorting emails into spam or inbox

Detecting positive or negative reviews

Classifying news articles by topic

Filtering harmful comments online

Organizing customer feedback by category

Syntax

TensorFlow

model = tf.keras.Sequential([
    tf.keras.layers.Embedding(input_dim=vocab_size, output_dim=embedding_dim, input_length=max_length),
    tf.keras.layers.GlobalAveragePooling1D(),
    tf.keras.layers.Dense(units=24, activation='relu'),
    tf.keras.layers.Dense(units=num_classes, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

The Embedding layer turns words into numbers the model can understand.

Softmax activation in the last layer is used for multi-class classification.

Examples

A simple model for classifying text into 2 categories.

TensorFlow

model = tf.keras.Sequential([
    tf.keras.layers.Embedding(1000, 16, input_length=100),
    tf.keras.layers.GlobalAveragePooling1D(),
    tf.keras.layers.Dense(16, activation='relu'),
    tf.keras.layers.Dense(2, activation='softmax')
])

Compile the model with Adam optimizer and accuracy metric.

TensorFlow

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

Sample Model

This program trains a simple text classifier to label sentences as positive or negative. It uses tokenization, padding, and a small neural network. Then it predicts labels for new sentences.

TensorFlow

import tensorflow as tf
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

# Sample data
sentences = [
    'I love machine learning',
    'Deep learning is fun',
    'I hate spam emails',
    'Spam emails are annoying',
    'I enjoy reading books',
    'Books are great for learning'
]
labels = [1, 1, 0, 0, 1, 1]  # 1 = positive, 0 = negative

# Parameters
vocab_size = 50
embedding_dim = 16
max_length = 5
trunc_type = 'post'

# Tokenize and pad
tokenizer = Tokenizer(num_words=vocab_size, oov_token='<OOV>')
tokenizer.fit_on_texts(sentences)
sequences = tokenizer.texts_to_sequences(sentences)
padded = pad_sequences(sequences, maxlen=max_length, truncating=trunc_type)

# Build model
model = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),
    tf.keras.layers.GlobalAveragePooling1D(),
    tf.keras.layers.Dense(24, activation='relu'),
    tf.keras.layers.Dense(2, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Train model
history = model.fit(padded, labels, epochs=10, verbose=0)

# Predict
test_sentences = ['I love books', 'Spam is bad']
test_seq = tokenizer.texts_to_sequences(test_sentences)
test_pad = pad_sequences(test_seq, maxlen=max_length, truncating=trunc_type)
predictions = model.predict(test_pad)

# Print predictions
for i, sentence in enumerate(test_sentences):
    pred_label = predictions[i].argmax()
    print(f"'{sentence}' classified as: {'Positive' if pred_label == 1 else 'Negative'}")

Important Notes

Make sure to preprocess text by tokenizing and padding so all inputs have the same length.

Use 'sparse_categorical_crossentropy' loss when labels are integers, not one-hot vectors.

Training with more data improves accuracy significantly.

Summary

Text classification sorts text into categories using a neural network.

Tokenize and pad text before feeding it to the model.

Use embedding layers and softmax output for multi-class problems.

Practice

1. What is the main purpose of a text classification model in TensorFlow?

easy

A. To sort text into different categories based on content

B. To generate new text from a given input

C. To translate text from one language to another

D. To detect objects in images

Solution

1. Step 1: Understand the task of text classification

   Text classification means assigning labels or categories to text data based on its content.

2. Step 2: Match the task to the options

   To sort text into different categories based on content describes sorting text into categories, which matches text classification.

3. Final Answer:

   To sort text into different categories based on content -> Option A

4. Quick Check:

   Text classification = sorting text [OK]

Hint: Text classification means sorting text into groups [OK]

Common Mistakes:

• Confusing classification with text generation
• Thinking classification is for images
• Mixing up translation with classification

2. Which of the following is the correct way to add an embedding layer in a TensorFlow Sequential model for text classification?

easy

A. model.add(tf.keras.layers.Dense(64, input_dim=1000))

B. model.add(tf.keras.layers.Conv2D(64, kernel_size=3))

C. model.add(tf.keras.layers.Embedding(input_dim=1000, output_dim=64))

D. model.add(tf.keras.layers.LSTM(64, input_shape=(1000,)))

Solution

1. Step 1: Identify the embedding layer syntax

   The embedding layer in TensorFlow is added with tf.keras.layers.Embedding and requires input_dim and output_dim parameters.

2. Step 2: Check each option

   model.add(tf.keras.layers.Embedding(input_dim=1000, output_dim=64)) correctly uses Embedding with input_dim=1000 and output_dim=64. Others use Dense, Conv2D, or LSTM which are not embedding layers.

3. Final Answer:

   model.add(tf.keras.layers.Embedding(input_dim=1000, output_dim=64)) -> Option C

4. Quick Check:

   Embedding layer uses tf.keras.layers.Embedding [OK]

Hint: Embedding layer uses tf.keras.layers.Embedding with input/output dims [OK]

Common Mistakes:

• Using Dense or Conv2D instead of Embedding
• Missing input_dim or output_dim parameters
• Confusing LSTM with embedding layer

3. Given the following TensorFlow model code for text classification, what will be the shape of the output layer's predictions for a batch of 32 samples if there are 5 classes?

model = tf.keras.Sequential([
  tf.keras.layers.Embedding(input_dim=1000, output_dim=16, input_length=10),
  tf.keras.layers.GlobalAveragePooling1D(),
  tf.keras.layers.Dense(5, activation='softmax')
])

predictions = model(tf.random.uniform((32, 10), maxval=1000, dtype=tf.int32))

medium

A. (10, 5)

B. (32, 5)

C. (32, 10)

D. (5, 32)

Solution

1. Step 1: Analyze model output layer

   The last Dense layer has 5 units with softmax, so output shape per sample is (5,).

2. Step 2: Consider batch size

   Input batch size is 32, so output shape is (32, 5).

3. Final Answer:

   (32, 5) -> Option B

4. Quick Check:

   Batch size 32 x 5 classes = (32, 5) [OK]

Hint: Output shape = (batch size, number of classes) [OK]

Common Mistakes:

• Confusing sequence length with batch size
• Mixing up input and output shapes
• Ignoring batch dimension in output

4. You wrote this TensorFlow model for text classification but get a shape error during training:

model = tf.keras.Sequential([
  tf.keras.layers.Embedding(input_dim=5000, output_dim=32),
  tf.keras.layers.Dense(10, activation='relu'),
  tf.keras.layers.Dense(3, activation='softmax')
])

What is the likely cause of the error?

medium

A. Dense layers cannot follow Embedding layers

B. Using wrong activation function in the last layer

C. input_dim in Embedding is too small

D. Missing a Flatten or GlobalAveragePooling1D layer after Embedding

Solution

1. Step 1: Understand Embedding output shape

   Embedding outputs 3D tensor (batch, sequence_length, embedding_dim). Dense expects 2D input.

2. Step 2: Identify missing layer

   Without Flatten or GlobalAveragePooling1D, Dense layer gets wrong input shape causing error.

3. Final Answer:

   Missing a Flatten or GlobalAveragePooling1D layer after Embedding -> Option D

4. Quick Check:

   Embedding output must be flattened before Dense [OK]

Hint: Add pooling or flatten after Embedding before Dense layers [OK]

Common Mistakes:

• Assuming Dense accepts 3D input
• Changing activation won't fix shape error
• Believing Dense can't follow Embedding at all

5. You want to build a text classification model in TensorFlow that handles variable-length input sentences and classifies them into 4 categories. Which model design is best?

hard

A. Use Embedding layer, GlobalAveragePooling1D, then Dense with 4 units and softmax activation

B. Use Embedding layer, Flatten, then Dense with 4 units and sigmoid activation

C. Use Embedding layer, Conv2D, then Dense with 1 unit and softmax activation

D. Use Dense layers only without Embedding

Solution

1. Step 1: Handle variable-length input

   GlobalAveragePooling1D can convert variable-length sequences to fixed-size vectors.

2. Step 2: Choose output layer for multi-class classification

   Dense with 4 units and softmax activation outputs probabilities for 4 classes.

3. Final Answer:

   Use Embedding layer, GlobalAveragePooling1D, then Dense with 4 units and softmax activation -> Option A

4. Quick Check:

   Pooling + softmax = best for variable-length multi-class [OK]

Hint: Use pooling + softmax for variable-length multi-class text [OK]

Common Mistakes:

• Using Flatten breaks variable-length input handling
• Using sigmoid for multi-class instead of softmax
• Using Conv2D for text sequences incorrectly