Handwriting recognition basics in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Handwriting recognition basics

Which metric matters for handwriting recognition and WHY

For handwriting recognition, accuracy is important because it shows how many characters or words the model gets right. But accuracy alone can be misleading if some characters appear much more often than others. So, we also look at precision and recall to understand how well the model finds the correct characters without mistakes or misses. For example, recall tells us if the model misses some letters, and precision tells us if the model wrongly guesses letters. The F1 score balances both precision and recall, giving a clear picture of overall performance.

Confusion matrix example for handwriting recognition

      Confusion Matrix (for 3 characters: A, B, C)

          Predicted
          A   B   C
    A   50   2   3
    B    4  45   1
    C    2   3  48

    Explanation:
    - True Positives (TP) for 'A' = 50 (correctly recognized A)
    - False Positives (FP) for 'A' = 4 + 2 = 6 (B and C wrongly predicted as A)
    - False Negatives (FN) for 'A' = 2 + 3 = 5 (A wrongly predicted as B or C)
    - True Negatives (TN) = total samples - TP - FP - FN
    

Precision vs Recall tradeoff with examples

Imagine the model is recognizing handwritten letters:

• High Precision: The model rarely mistakes one letter for another. For example, when it says a letter is 'A', it is almost always correct. This is good if you want to avoid wrong letters in official documents.
• High Recall: The model finds almost all the 'A's in the text, even if it sometimes mistakes other letters for 'A'. This is important if missing a letter is worse than a few mistakes, like reading a handwritten note where every letter matters.

Balancing precision and recall depends on what is more important: avoiding mistakes or not missing letters.

What good vs bad metric values look like for handwriting recognition

• Good: Accuracy above 90%, precision and recall both above 85%, and F1 score close to 0.9. This means the model correctly recognizes most letters and rarely makes mistakes or misses letters.
• Bad: Accuracy below 70%, precision or recall below 50%. This means the model often mistakes letters or misses many letters, making the recognition unreliable.

Common pitfalls in handwriting recognition metrics

• Accuracy paradox: If some letters appear very often, a model guessing only those letters can have high accuracy but poor real performance.
• Data leakage: If the model sees the same handwriting style in training and testing, it may look better than it really is.
• Overfitting: The model performs very well on training data but poorly on new handwriting styles.
• Ignoring class imbalance: Some letters appear less often, so metrics should consider this to avoid misleading results.

Self-check question

Your handwriting recognition model has 98% accuracy but only 12% recall on the letter 'A'. Is it good for production? Why or why not?

Answer: No, it is not good. Even though accuracy is high, the model misses most 'A's (low recall). This means many 'A's are not recognized, which can cause serious errors in reading handwriting.

Key Result

For handwriting recognition, balanced precision and recall with high accuracy ensure the model correctly identifies letters without missing or wrongly predicting them.

Practice

1. What is the main goal of handwriting recognition in computer vision?

easy

A. To convert images of handwritten text into digital text

B. To create handwritten images from typed text

C. To detect faces in handwritten notes

D. To enhance the colors of handwritten images

Solution

1. Step 1: Understand handwriting recognition purpose

   Handwriting recognition aims to read and convert handwritten text images into machine-readable text.

2. Step 2: Compare options with this goal

   Only To convert images of handwritten text into digital text matches this goal; others describe unrelated tasks.

3. Final Answer:

   To convert images of handwritten text into digital text -> Option A

4. Quick Check:

   Handwriting recognition = convert handwriting to text [OK]

Hint: Think: handwriting recognition means reading handwriting [OK]

Common Mistakes:

• Confusing recognition with image enhancement
• Thinking it creates handwriting instead of reading it
• Mixing handwriting with face detection

2. Which Python library is commonly used to load the MNIST dataset for handwriting recognition?

easy

A. pandas

B. matplotlib

C. tensorflow.keras.datasets

D. scikit-learn.preprocessing

Solution

1. Step 1: Recall common MNIST loading methods

   The MNIST dataset is often loaded using tensorflow.keras.datasets for easy access.

2. Step 2: Check options for dataset loading

   Only tensorflow.keras.datasets provides direct MNIST loading; others do not.

3. Final Answer:

   tensorflow.keras.datasets -> Option C

4. Quick Check:

   MNIST load = tensorflow.keras.datasets [OK]

Hint: Remember: TensorFlow has built-in MNIST loader [OK]

Common Mistakes:

• Choosing matplotlib which is for plotting
• Selecting pandas which handles tables, not images
• Confusing preprocessing with dataset loading

3. What will be the output shape of the images array after loading MNIST dataset with (x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()?

medium

A. (28, 28, 60000)

B. (60000, 28, 28)

C. (60000, 784)

D. (60000, 28, 28, 1)

Solution

1. Step 1: Understand MNIST image shape

   MNIST images are 28x28 pixels grayscale images, and training set has 60000 samples.

2. Step 2: Check output shape from load_data()

   Images are loaded as (60000, 28, 28) without channel dimension by default.

3. Final Answer:

   (60000, 28, 28) -> Option B

4. Quick Check:

   MNIST images shape = (60000, 28, 28) [OK]

Hint: MNIST images are 28x28 pixels, 60000 training samples [OK]

Common Mistakes:

• Assuming images are flattened to 784 by default
• Confusing channel dimension presence
• Mixing sample count with image dimensions

4. Identify the error in this simple neural network code for handwriting recognition:

model = tf.keras.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28, 1)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dense(10)
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

medium

A. Optimizer name is invalid

B. Missing activation function in the last Dense layer

C. Wrong loss function for classification

D. Incorrect input_shape in Flatten layer

Solution

1. Step 1: Review model architecture

   MNIST images from load_data() have shape (60000, 28, 28).

2. Step 2: Check input_shape in Flatten

   input_shape=(28, 28, 1) expects input of shape (None, 28, 28, 1), but MNIST data is (None, 28, 28), causing shape mismatch.

3. Final Answer:

   Incorrect input_shape in Flatten layer -> Option D

4. Quick Check:

   MNIST x_train.shape = (60000, 28, 28), input_shape=(28, 28) [OK]

Hint: MNIST default shape is (60000, 28, 28), no channel dim [OK]

Common Mistakes:

• Focusing on missing output activation (optional with this loss)
• Thinking loss is wrong (correct for integer labels)
• Assuming optimizer string is invalid (strings work)

5. You want to improve handwriting recognition accuracy by adding dropout to the model. Which code snippet correctly adds dropout after the first Dense layer?

hard

A. tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dropout(0.2)

B. tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(128, activation='relu')

C. tf.keras.layers.Dense(128, activation='relu', dropout=0.2)

D. tf.keras.layers.Dense(128, activation='relu', rate=0.2)

Solution

1. Step 1: Understand dropout usage in Keras

   Dropout is a separate layer added after a Dense layer to randomly ignore neurons during training.

2. Step 2: Check each option for correct syntax

   tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dropout(0.2) correctly places Dropout after Dense with correct parameter 0.2; options C and D incorrectly add dropout as Dense parameters; tf.keras.layers.Dropout(0.2), tf.keras.layers.Dense(128, activation='relu') reverses order, which is not standard.

3. Final Answer:

   tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dropout(0.2) -> Option A

4. Quick Check:

   Dropout is a separate layer after Dense [OK]

Hint: Dropout is its own layer placed after Dense layer [OK]

Common Mistakes:

• Trying to add dropout as Dense layer argument
• Placing Dropout before Dense layer
• Using wrong parameter names for dropout