Text detection in images in Computer Vision - ML Experiment: Train & Evaluate

Experiment - Text detection in images

Problem:Detect and locate text regions in images using a deep learning model.

Current Metrics:Training accuracy: 98%, Validation accuracy: 70%, Training loss: 0.05, Validation loss: 0.35

Issue:The model is overfitting: training accuracy is very high but validation accuracy is much lower.

Your Task

Reduce overfitting to improve validation accuracy to at least 85% while keeping training accuracy below 92%.

You can only modify the model architecture and training hyperparameters.

Do not change the dataset or input image preprocessing.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Define data augmentation
train_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=10,
    width_shift_range=0.1,
    height_shift_range=0.1,
    shear_range=0.1,
    zoom_range=0.1,
    horizontal_flip=True,
    fill_mode='nearest'
)

val_datagen = ImageDataGenerator(rescale=1./255)

# Assume train_generator and val_generator are created from directories using train_datagen and val_datagen respectively

# Build model with dropout
model = models.Sequential([
    layers.Conv2D(32, (3,3), activation='relu', input_shape=(128,128,3)),
    layers.MaxPooling2D(2,2),
    layers.Dropout(0.25),
    layers.Conv2D(64, (3,3), activation='relu'),
    layers.MaxPooling2D(2,2),
    layers.Dropout(0.25),
    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(1, activation='sigmoid')
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0005),
              loss='binary_crossentropy',
              metrics=['accuracy'])

# Early stopping callback
early_stop = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

# Train model
history = model.fit(
    train_generator,
    epochs=50,
    validation_data=val_generator,
    callbacks=[early_stop]
)

Added dropout layers after convolutional and dense layers to reduce overfitting.

Implemented data augmentation to increase training data variety.

Reduced learning rate from default to 0.0005 for smoother training.

Added early stopping to stop training when validation loss stops improving.

Results Interpretation

Before: Training accuracy 98%, Validation accuracy 70%, Training loss 0.05, Validation loss 0.35

After: Training accuracy 90%, Validation accuracy 87%, Training loss 0.18, Validation loss 0.25

Adding dropout and data augmentation helps reduce overfitting, improving validation accuracy and making the model generalize better to new images.

Bonus Experiment

Try using a pre-trained model like MobileNetV2 as a feature extractor for text detection and fine-tune it.

💡 Hint

Use transfer learning by freezing the base model layers and training only the top layers first, then unfreeze some layers for fine-tuning.

Practice

1. What is the main goal of text detection in images?

easy

A. To find where text appears in an image

B. To translate text from one language to another

C. To change the font style of text in images

D. To remove text from images

Solution

1. Step 1: Understand the purpose of text detection

   Text detection means locating the areas in an image that contain text.

2. Step 2: Differentiate from other text-related tasks

   Tasks like translation or font change happen after detecting text, not during detection.

3. Final Answer:

   To find where text appears in an image -> Option A

4. Quick Check:

   Text detection = locating text [OK]

Hint: Text detection means locating text areas in images [OK]

Common Mistakes:

• Confusing detection with translation
• Thinking detection changes text style
• Assuming detection removes text

2. Which Python library is commonly used for text detection and recognition in images?

easy

A. pytesseract

B. matplotlib

C. numpy

D. scikit-learn

Solution

1. Step 1: Identify libraries related to text detection

   pytesseract is a Python wrapper for Tesseract OCR, used for detecting and reading text.

2. Step 2: Exclude unrelated libraries

   matplotlib is for plotting, numpy for arrays, scikit-learn for general ML, not specific to text detection.

3. Final Answer:

   pytesseract -> Option A

4. Quick Check:

   pytesseract = text detection tool [OK]

Hint: pytesseract is the go-to for OCR in Python [OK]

Common Mistakes:

• Choosing matplotlib for text detection
• Confusing numpy with OCR tools
• Selecting scikit-learn for image text reading

3. What will the following Python code output if image_path contains a clear text image?

import pytesseract
from PIL import Image
img = Image.open(image_path)
text = pytesseract.image_to_string(img)
print(text.strip())

medium

A. An error because pytesseract cannot open images

B. The text content found in the image

C. The image object details printed

D. An empty string always

Solution

1. Step 1: Understand the code flow

   The code opens an image, uses pytesseract to extract text, then prints the text without extra spaces.

2. Step 2: Predict output for a clear text image

   Since the image has clear text, pytesseract returns that text as a string, which is printed.

3. Final Answer:

   The text content found in the image -> Option B

4. Quick Check:

   pytesseract extracts text string [OK]

Hint: pytesseract.image_to_string returns detected text [OK]

Common Mistakes:

• Expecting an error from pytesseract
• Thinking it prints image object info
• Assuming output is always empty

4. Identify the error in this code snippet for detecting text in an image:

import pytesseract
img = 'image.jpg'
text = pytesseract.image_to_string(img)
print(text)

medium

A. Using print instead of return

B. Missing import for PIL Image

C. No error, code runs fine

D. Passing a string filename instead of an image object

Solution

1. Step 1: Check input type for pytesseract.image_to_string

   This function accepts both a PIL Image object and a filename string as input.

2. Step 2: Verify the code

   The code passes a string filename ('image.jpg'), which is valid, so no error occurs and it will extract text if the file exists.

3. Final Answer:

   No error, code runs fine -> Option C

4. Quick Check:

   image_to_string accepts string path [OK]

Hint: pytesseract.image_to_string accepts filename paths directly [OK]

Common Mistakes:

• Thinking print should be return
• Assuming PIL Image import is required
• Believing only image objects are accepted

5. You want to detect text in a photo with multiple languages. Which approach is best to improve accuracy?

hard

A. Use only English language setting

B. Convert image to grayscale only

C. Resize image to a smaller size

D. Specify all target languages in pytesseract's config parameter

Solution

1. Step 1: Understand multi-language text detection

   pytesseract supports multiple languages by specifying them in the config parameter.

2. Step 2: Evaluate other options

   Grayscale conversion helps but doesn't handle languages; resizing smaller reduces detail; English-only misses other languages.

3. Final Answer:

   Specify all target languages in pytesseract's config parameter -> Option D

4. Quick Check:

   Multi-language config improves detection [OK]

Hint: Use config to set multiple languages in pytesseract [OK]

Common Mistakes:

• Ignoring language settings
• Reducing image size too much
• Assuming grayscale alone solves language issues