Tesseract OCR in Computer Vision - ML Experiment: Train & Evaluate

Experiment - Tesseract OCR

Problem:Extract text from images using Tesseract OCR but the current model misses some characters and produces errors.

Current Metrics:Character accuracy: 75%, Word accuracy: 60%

Issue:The OCR output has many mistakes due to noisy images and lack of preprocessing.

Your Task

Improve OCR accuracy to at least 85% character accuracy and 75% word accuracy by reducing noise and improving image quality before OCR.

Must use Tesseract OCR for text extraction.

Can only modify image preprocessing steps before OCR.

No changes to Tesseract internal settings or training.

Hint 1

Hint 2

Hint 3

Solution

Computer Vision

import cv2
import pytesseract

# Load image
image = cv2.imread('sample_text_image.png')

# Convert to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Apply thresholding to get binary image
_, thresh = cv2.threshold(gray, 150, 255, cv2.THRESH_BINARY_INV)

# Remove noise with dilation and erosion
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,1))
dilated = cv2.dilate(thresh, kernel, iterations=1)
eroded = cv2.erode(dilated, kernel, iterations=1)

# Resize image to double size for better OCR
resized = cv2.resize(eroded, None, fx=2, fy=2, interpolation=cv2.INTER_LINEAR)

# Invert image back for Tesseract (white background)
processed = cv2.bitwise_not(resized)

# OCR extraction
text = pytesseract.image_to_string(processed, lang='eng')

print('Extracted Text:')
print(text)

Converted image to grayscale to simplify colors.

Applied thresholding to create a clear black and white image.

Used dilation and erosion to reduce noise and improve character shapes.

Resized image to double the original size to help Tesseract read characters better.

Inverted image colors to match Tesseract's expected input (black text on white background).

Results Interpretation

Before: Character accuracy 75%, Word accuracy 60%
After: Character accuracy 88%, Word accuracy 78%

Proper image preprocessing like grayscale conversion, thresholding, noise removal, and resizing can significantly improve OCR accuracy without changing the OCR engine itself.

Bonus Experiment

Try using adaptive thresholding instead of fixed thresholding to handle images with uneven lighting.

💡 Hint

Use cv2.adaptiveThreshold with parameters tuned for your image to improve text visibility.

Practice

1. What is the main purpose of Tesseract OCR in computer vision?

easy

A. To enhance image resolution

B. To detect objects in images

C. To convert images containing text into editable text

D. To classify images into categories

Solution

1. Step 1: Understand Tesseract OCR's function

   Tesseract OCR is designed to read text from images and convert it into editable text format.

2. Step 2: Compare options with Tesseract's purpose

   Image enhancement, object detection, and image classification relate to other computer vision tasks but not text extraction, which is Tesseract's main use.

3. Final Answer:

   To convert images containing text into editable text -> Option C

4. Quick Check:

   Tesseract OCR = Text extraction [OK]

Hint: Remember OCR means Optical Character Recognition [OK]

Common Mistakes:

• Confusing OCR with image enhancement
• Thinking Tesseract detects objects
• Assuming it classifies images

2. Which Python function is used to extract text from an image using Tesseract?

easy

A. pytesseract.image_to_string()

B. pytesseract.extract_text()

C. pytesseract.read_image()

D. pytesseract.text_from_image()

Solution

1. Step 1: Recall the correct pytesseract function

   The official function to get text from an image is image_to_string().

2. Step 2: Verify other options

   Other options are not valid pytesseract functions and will cause errors.

3. Final Answer:

   pytesseract.image_to_string() -> Option A

4. Quick Check:

   Function for text extraction = image_to_string() [OK]

Hint: Use image_to_string() to get text from images [OK]

Common Mistakes:

• Using non-existent pytesseract functions
• Confusing function names with similar words
• Forgetting parentheses in function call

3. What will be the output of this Python code snippet using pytesseract?

from PIL import Image
import pytesseract
img = Image.new('RGB', (100, 30), color = (255, 255, 255))
text = pytesseract.image_to_string(img)
print(text.strip())

medium

A. Random characters

B. Empty string

C. Error: Image not found

D. Whitespace characters

Solution

1. Step 1: Analyze the image content

   The image is blank white with no text drawn on it.

2. Step 2: Understand pytesseract output on blank images

   Since no text exists, pytesseract returns an empty string or whitespace which is stripped to empty.

3. Final Answer:

   Empty string -> Option B

4. Quick Check:

   Blank image text output = empty string [OK]

Hint: Blank images give empty text output [OK]

Common Mistakes:

• Expecting error due to no text
• Assuming random characters appear
• Not stripping whitespace before print

4. Identify the error in this code snippet using pytesseract:

import pytesseract
text = pytesseract.image_to_string('image.png')
print(text)

medium

A. No error, code runs fine

B. Missing import for PIL Image

C. Incorrect function name used

D. Passing a filename string instead of an image object

Solution

1. Step 1: Check function argument requirements

   image_to_string() accepts both PIL Image objects and strings representing image file paths.

2. Step 2: Verify the code

   Passing a filename string 'image.png' is valid assuming the file exists and pytesseract is configured.

3. Final Answer:

   No error, code runs fine -> Option A

4. Quick Check:

   image_to_string() accepts file paths [OK]

Hint: pytesseract.image_to_string() accepts both image objects and file paths [OK]

Common Mistakes:

• Thinking only PIL Image objects are accepted
• Assuming PIL import is required for file paths
• Believing the function cannot read files directly

5. You want to improve Tesseract OCR accuracy on a scanned document image with noise and skew. Which combination of preprocessing steps is best before using pytesseract.image_to_string()?

hard

A. Apply random color filters

B. Increase image brightness only

C. Resize image to smaller dimensions

D. Convert to grayscale, apply thresholding, and deskew the image

Solution

1. Step 1: Understand common OCR preprocessing

   Grayscale conversion simplifies colors, thresholding makes text clearer, and deskew corrects tilted text improving OCR accuracy.

2. Step 2: Evaluate other options

   Increasing brightness alone or resizing smaller can reduce quality; random color filters add noise, hurting OCR.

3. Final Answer:

   Convert to grayscale, apply thresholding, and deskew the image -> Option D

4. Quick Check:

   Preprocessing for OCR = grayscale + threshold + deskew [OK]

Hint: Clean and straighten image before OCR for best results [OK]

Common Mistakes:

• Skipping deskewing step
• Using color filters that add noise
• Reducing image size too much