Table extraction from images in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Table extraction from images

Which metric matters for Table extraction from images and WHY

For table extraction, we want to measure how well the model finds the correct table cells and their content. Key metrics include Precision and Recall. Precision tells us how many detected table cells are actually correct, avoiding false detections. Recall tells us how many real table cells the model found, avoiding missed cells. The F1 score balances these two. High precision means clean, accurate tables; high recall means complete tables. We also use Intersection over Union (IoU) to check how well the predicted cell boxes overlap with the true boxes.

Confusion matrix for Table extraction

True Positives (TP): Correctly detected table cells
False Positives (FP): Detected cells that are not real
False Negatives (FN): Real cells missed by the model

Example confusion matrix counts:
+----------------+----------------+
|                | Predicted Cell |
|                | Yes      No    |
+----------------+----------------+
| Actual Cell Yes| TP = 80  FN = 20|
| Actual Cell No | FP = 10  TN = 90|
+----------------+----------------+

Total cells = TP + FP + FN + TN = 80 + 10 + 20 + 90 = 200

Precision = TP / (TP + FP) = 80 / (80 + 10) = 0.89
Recall = TP / (TP + FN) = 80 / (80 + 20) = 0.80
F1 Score = 2 * (0.89 * 0.80) / (0.89 + 0.80) ≈ 0.84
    

Precision vs Recall tradeoff with examples

If the model has high precision but low recall, it means it finds mostly correct table cells but misses many real ones. This leads to incomplete tables, which can be bad if you need full data.

If the model has high recall but low precision, it finds most real cells but also includes many wrong ones. This creates noisy tables with errors.

For example, in financial reports, missing a table cell (low recall) can lose important data. But including wrong cells (low precision) can cause wrong calculations. So a balance (high F1) is best.

What good vs bad metric values look like for Table extraction

• Good: Precision and Recall above 0.85, F1 score above 0.85, IoU above 0.75 for cell bounding boxes. This means most cells are correctly found and well localized.
• Bad: Precision or Recall below 0.5 means many errors or misses. F1 below 0.6 means poor balance. IoU below 0.5 means boxes do not match well, causing wrong cell boundaries.

Common pitfalls in Table extraction metrics

• Accuracy paradox: If most images have no tables, a model that always predicts no table can have high accuracy but is useless.
• Data leakage: Using the same documents for training and testing inflates metrics falsely.
• Overfitting: Very high training metrics but low test metrics means the model memorizes tables instead of generalizing.
• Ignoring IoU: Counting a detected cell as correct without checking overlap can overestimate performance.

Self-check question

Your table extraction model has 98% accuracy but only 12% recall on table cells. Is it good for production? Why or why not?

Answer: No, it is not good. The high accuracy is misleading because most images or areas may not have tables, so predicting no table often is easy. The very low recall means the model misses almost all real table cells, so it fails to extract useful tables. For production, you need much higher recall to capture the tables fully.

Key Result

Precision and recall are key to measure correct and complete table cell detection; balance them for best extraction quality.

Practice

1. What is the main goal of table extraction from images in computer vision?

easy

A. Create new tables from scratch

B. Convert images of tables into editable and structured data

C. Enhance the colors of table images

D. Compress table images to save space

Solution

1. Step 1: Understand the purpose of table extraction

   Table extraction aims to transform images containing tables into a format that can be edited and analyzed, such as spreadsheets.

2. Step 2: Compare options to the goal

   Options A, B, and D do not relate to converting image content into editable data, but C does.

3. Final Answer:

   Convert images of tables into editable and structured data -> Option B

4. Quick Check:

   Table extraction = Editable data from images [OK]

Hint: Focus on converting images to editable data [OK]

Common Mistakes:

• Confusing image enhancement with data extraction
• Thinking table extraction creates tables from nothing
• Assuming compression is the goal

2. Which of the following is the correct step to start table extraction from an image using Python libraries?

easy

A. Use OCR to read text directly without detecting table structure

B. Resize the image to a smaller size and save it

C. Detect table boundaries and cells before applying OCR

D. Apply color filters to change table colors

Solution

1. Step 1: Identify the correct workflow for table extraction

   First, detecting the table structure (boundaries and cells) is essential to know where text is located.

2. Step 2: Understand the role of OCR

   OCR reads text inside detected cells after structure detection, so applying OCR first is incorrect.

3. Final Answer:

   Detect table boundaries and cells before applying OCR -> Option C

4. Quick Check:

   Detect structure first, then OCR [OK]

Hint: Detect table layout before reading text [OK]

Common Mistakes:

• Applying OCR before detecting table cells
• Focusing on image color changes instead of structure
• Skipping structure detection

3. Given the following Python snippet using OpenCV and pytesseract for table extraction, what will be the output type of cells_text?

import cv2
import pytesseract

image = cv2.imread('table.png', 0)
_, thresh = cv2.threshold(image, 128, 255, cv2.THRESH_BINARY_INV)
contours, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cells_text = []
for cnt in contours:
    x, y, w, h = cv2.boundingRect(cnt)
    cell_img = image[y:y+h, x:x+w]
    text = pytesseract.image_to_string(cell_img, config='--psm 6')
    cells_text.append(text.strip())
print(type(cells_text))

medium

A.

B.

C.

D.

Solution

1. Step 1: Analyze the code snippet

   The variable cells_text is initialized as an empty list and text from each detected cell is appended to it.

2. Step 2: Determine the type of cells_text

   Since cells_text collects multiple strings in a list, its type remains list.

3. Final Answer:

   <class 'list'> -> Option A

4. Quick Check:

   Appending text to list = list type [OK]

Hint: Check variable initialization and append usage [OK]

Common Mistakes:

• Confusing the output of print(type())
• Assuming OCR returns a dict or int
• Ignoring the list append operation

4. You run a table extraction pipeline but notice that some table cells are merged incorrectly, causing wrong text grouping. What is the most likely cause?

medium

A. Incorrect contour detection merging nearby cells

B. OCR engine misreading characters inside cells

C. Image color enhancement applied before extraction

D. Saving the output file in wrong format

Solution

1. Step 1: Identify the problem source

   Merged cells usually happen when contour detection groups multiple cells as one shape.

2. Step 2: Rule out other options

   OCR misreading affects text accuracy but not cell merging. Color enhancement and file format do not cause merging issues.

3. Final Answer:

   Incorrect contour detection merging nearby cells -> Option A

4. Quick Check:

   Cell merging = contour detection error [OK]

Hint: Check contour detection for cell boundaries [OK]

Common Mistakes:

• Blaming OCR for cell merging
• Ignoring image preprocessing effects
• Assuming file format affects cell detection

5. You want to extract tables from scanned invoices with varying layouts. Which approach best improves accuracy of table extraction?

hard

A. Apply fixed thresholding and contour detection without training

B. Manually crop each table region before extraction

C. Use only OCR on the full invoice image without detecting tables

D. Train a deep learning model to detect table structures and cells before OCR

Solution

1. Step 1: Understand the challenge of varying layouts

   Invoices have different table styles, so fixed rules may fail to detect tables accurately.

2. Step 2: Evaluate approaches for adaptability

   Training a deep learning model can learn diverse table structures and generalize better than fixed methods or manual cropping.

3. Final Answer:

   Train a deep learning model to detect table structures and cells before OCR -> Option D

4. Quick Check:

   Varying layouts = train model for detection [OK]

Hint: Use learning models for diverse table layouts [OK]

Common Mistakes:

• Relying on fixed thresholding for all layouts
• Skipping table detection and using only OCR
• Manual cropping is not scalable