MediaPipe Pose in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - MediaPipe Pose

Which metric matters for MediaPipe Pose and WHY

For MediaPipe Pose, the key metric is Mean Average Precision (mAP) or Percentage of Correct Keypoints (PCK). These metrics measure how accurately the model detects body landmarks compared to the true positions.

We care about these because the goal is to find exact points on the body like elbows or knees. If the points are off, the pose estimation is wrong. So, accuracy in locating these points is critical.

Also, inference speed matters because pose detection often runs live on video. A slow model makes the experience laggy.

Confusion matrix or equivalent visualization

Pose estimation does not use a classic confusion matrix because it predicts many points per image. Instead, we use a distance threshold to decide if a predicted keypoint is correct.

    True Keypoint Positions:    (x1, y1), (x2, y2), ...
    Predicted Keypoint Positions: (x1', y1'), (x2', y2'), ...

    For each keypoint:
      If distance(predicted, true) < threshold: count as True Positive (TP)
      Else: False Positive (FP) or False Negative (FN) depending on missing points

    Total keypoints = TP + FP + FN
    

This helps calculate Precision, Recall, and F1 score for keypoint detection.

Precision vs Recall tradeoff with examples

Precision means how many detected keypoints are actually correct. High precision means few false points.

Recall means how many true keypoints the model found. High recall means few missed points.

For example, in a fitness app, missing a keypoint (low recall) can cause wrong exercise feedback. So recall is very important.

But if the model detects many wrong points (low precision), the app might confuse the user. So precision also matters.

We balance precision and recall to get a good overall F1 score, ensuring the model finds most points and keeps them accurate.

What "good" vs "bad" metric values look like for MediaPipe Pose

Good values:

• Precision > 0.85 (85%) - Most detected points are correct
• Recall > 0.85 (85%) - Most true points are found
• F1 score > 0.85 - Balanced and accurate detection
• Inference speed < 30 ms per frame - Real-time performance

Bad values:

• Precision < 0.6 - Many false points confuse the system
• Recall < 0.6 - Many true points missed, poor pose estimation
• F1 score < 0.6 - Overall poor detection quality
• Inference speed > 100 ms per frame - Laggy and unusable live

Common pitfalls in MediaPipe Pose metrics

• Ignoring speed: A model with high accuracy but slow speed is not practical for live pose detection.
• Overfitting: Model performs well on training videos but poorly on new people or backgrounds.
• Data leakage: Testing on the same videos used for training inflates accuracy falsely.
• Using accuracy alone: Accuracy can be misleading because many keypoints are easy to detect; focus on precision, recall, and F1.
• Threshold choice: Setting the distance threshold too loose or tight changes metric results unfairly.

Self-check question

Your MediaPipe Pose model has 98% accuracy but only 12% recall on keypoints. Is it good for production? Why or why not?

Answer: No, it is not good. The very low recall means the model misses most true keypoints, so it fails to detect the full pose. High accuracy alone is misleading because many keypoints might be absent or ignored. For pose estimation, recall is critical to find all body points.

Key Result

For MediaPipe Pose, balanced high precision and recall (above 85%) with fast inference speed are key to good pose detection.

Practice

1. What is the main purpose of MediaPipe Pose in computer vision?

easy

A. To classify objects like cars and animals

B. To recognize faces in photos

C. To detect and track human body landmarks in images or videos

D. To enhance image colors automatically

Solution

1. Step 1: Understand MediaPipe Pose functionality

   MediaPipe Pose is designed to find key points on the human body, like joints, in images or videos.

2. Step 2: Compare options with this function

   Only To detect and track human body landmarks in images or videos describes detecting and tracking body landmarks, which matches MediaPipe Pose's purpose.

3. Final Answer:

   To detect and track human body landmarks in images or videos -> Option C

4. Quick Check:

   MediaPipe Pose = Body landmarks detection [OK]

Hint: Remember: MediaPipe Pose = human body keypoints [OK]

Common Mistakes:

• Confusing pose detection with face recognition
• Thinking it classifies objects instead of body parts
• Assuming it edits or enhances images

2. Which of the following is the correct way to import MediaPipe Pose in Python?

easy

A. import mediapipe as mp
pose = mp.solutions.pose.Pose()

B. import mediapipe.pose as mp
pose = mp.Pose()

C. from mediapipe import pose
pose = pose.Pose()

D. import mp_pose
pose = mp_pose.Pose()

Solution

1. Step 1: Recall MediaPipe import structure

   MediaPipe is imported as 'mediapipe as mp', and pose is accessed via 'mp.solutions.pose'.

2. Step 2: Check each option's syntax

   import mediapipe as mp pose = mp.solutions.pose.Pose() correctly imports and creates a Pose object. Others use incorrect module names or import styles.

3. Final Answer:

   import mediapipe as mp pose = mp.solutions.pose.Pose() -> Option A

4. Quick Check:

   Correct import = import mediapipe as mp pose = mp.solutions.pose.Pose() [OK]

Hint: MediaPipe uses 'mp.solutions.pose' for pose module [OK]

Common Mistakes:

• Trying to import pose directly from mediapipe
• Using wrong module names like 'mp_pose'
• Incorrect import syntax causing errors

3. Given this code snippet using MediaPipe Pose, what will be the output type of results.pose_landmarks after processing an image?

medium

A. A list of (x, y, z) coordinates for each detected landmark

B. A protobuf object containing landmark data with x, y, z fields

C. A numpy array of shape (33, 3) with landmark coordinates

D. A dictionary with landmark names as keys and coordinates as values

Solution

1. Step 1: Understand MediaPipe Pose output format

   MediaPipe Pose returns landmarks as a protobuf object, not a simple list or dict.

2. Step 2: Analyze options for output type

   A protobuf object containing landmark data with x, y, z fields correctly states the output is a protobuf object with x, y, z fields for each landmark.

3. Final Answer:

   A protobuf object containing landmark data with x, y, z fields -> Option B

4. Quick Check:

   Pose landmarks output = protobuf object [OK]

Hint: MediaPipe Pose landmarks are protobuf objects, not plain lists [OK]

Common Mistakes:

• Assuming output is a simple list or numpy array
• Expecting a dictionary with landmark names
• Confusing protobuf with JSON or dict

4. You wrote this code to detect pose landmarks but get an error: AttributeError: 'NoneType' object has no attribute 'landmark'. What is the likely cause?

medium

A. The input image is empty or invalid, so no landmarks detected

B. You forgot to import mediapipe before using it

C. The Pose object was not created correctly

D. You used the wrong method name instead of 'process'

Solution

1. Step 1: Understand the error meaning

   The error means 'results.pose_landmarks' is None, so accessing 'landmark' fails.

2. Step 2: Identify why pose_landmarks is None

   This happens if the input image has no detectable person or is invalid, so no landmarks are found.

3. Final Answer:

   The input image is empty or invalid, so no landmarks detected -> Option A

4. Quick Check:

   None landmarks = invalid or empty image [OK]

Hint: Check if input image is valid to avoid None landmarks [OK]

Common Mistakes:

• Assuming import errors cause this specific AttributeError
• Thinking Pose object creation causes this error
• Confusing method names causing this error

5. You want to build a fitness app that counts squats using MediaPipe Pose. Which approach best helps detect a squat repetition?

hard

A. Count how many times the wrist moves up and down

B. Measure the distance between shoulders to detect squat depth

C. Use face landmarks to detect head movement during squats

D. Track the angle between hip, knee, and ankle landmarks to detect bending

Solution

1. Step 1: Identify key body parts for squat detection

   Squats involve bending knees and hips, so tracking angles at these joints is important.

2. Step 2: Evaluate options for relevance

   Track the angle between hip, knee, and ankle landmarks to detect bending uses angles between hip, knee, and ankle landmarks, which directly relate to squat movement.

3. Final Answer:

   Track the angle between hip, knee, and ankle landmarks to detect bending -> Option D

4. Quick Check:

   Squat detection = joint angle tracking [OK]

Hint: Use joint angles, not wrist or face, to detect squats [OK]

Common Mistakes:

• Tracking wrist or face landmarks unrelated to squats
• Measuring shoulder distance which doesn't reflect squat depth
• Ignoring joint angles that show bending