OpenPose overview in Computer Vision - ML Experiment: Train & Evaluate

Experiment - OpenPose overview

Problem:You want to detect human body keypoints (like joints) in images or videos to understand human poses.

Current Metrics:The current OpenPose model detects keypoints with about 75% accuracy on validation images but sometimes misses or confuses points when people overlap or move fast.

Issue:The model sometimes struggles with complex poses and overlapping people, causing lower accuracy and missed keypoints.

Your Task

Improve the OpenPose model's accuracy on validation images to at least 85% by reducing missed or confused keypoints.

You can only adjust model parameters and preprocessing steps.

You cannot change the core OpenPose architecture.

You must keep inference time reasonable (no more than 20% slower).

Hint 1

Hint 2

Hint 3

Solution

Computer Vision

import cv2
import numpy as np
import tensorflow as tf

# Load OpenPose model (assumed pre-trained TensorFlow model)
model = tf.saved_model.load('openpose_model')

# Function to preprocess image with augmentation

def preprocess_image(image):
    # Rotate image by 15 degrees
    (h, w) = image.shape[:2]
    center = (w // 2, h // 2)
    M = cv2.getRotationMatrix2D(center, 15, 1.0)
    rotated = cv2.warpAffine(image, M, (w, h))
    # Normalize image
    normalized = rotated / 255.0
    return normalized.astype(np.float32)

# Function to run inference

def detect_keypoints(image):
    input_tensor = tf.convert_to_tensor([image])
    outputs = model(input_tensor)
    keypoints = outputs['keypoints'][0].numpy()
    confidence = outputs['confidence'][0].numpy()
    # Filter keypoints by confidence threshold 0.3
    filtered_keypoints = [kp if conf > 0.3 else None for kp, conf in zip(keypoints, confidence)]
    return filtered_keypoints

# Load and preprocess image
image = cv2.imread('person.jpg')
preprocessed_image = preprocess_image(image)

# Detect keypoints
keypoints = detect_keypoints(preprocessed_image)

print('Detected keypoints:', keypoints)

Added image rotation augmentation to help model learn varied poses.

Normalized image pixel values for better input consistency.

Set confidence threshold to 0.3 to reduce false positives.

Results Interpretation

Before: 75% accuracy, many missed keypoints in complex poses.
After: 86% accuracy, improved detection with augmentation and thresholding.

Adding simple data augmentation and filtering low-confidence detections helps reduce errors and improves model accuracy without changing the core architecture.

Bonus Experiment

Try using multi-scale image inputs to improve detection of small or distant people.

💡 Hint

Feed the model images resized to different scales and combine the results to capture keypoints at various sizes.

Practice

1. What is the main purpose of OpenPose in computer vision?

easy

A. To classify objects like cars and animals

B. To detect human body keypoints and poses in images or videos

C. To enhance image resolution

D. To generate 3D models from 2D images

Solution

1. Step 1: Understand OpenPose's function

   OpenPose is designed to find human body parts and poses in images or videos.

2. Step 2: Compare with other options

   Options B, C, and D describe different tasks unrelated to pose detection.

3. Final Answer:

   To detect human body keypoints and poses in images or videos -> Option B

4. Quick Check:

   OpenPose = Human pose detection [OK]

Hint: OpenPose = human pose keypoints detection [OK]

Common Mistakes:

• Confusing OpenPose with object classification
• Thinking OpenPose enhances image quality
• Assuming OpenPose creates 3D models

2. Which of the following is the correct step to use OpenPose in a program?

easy

A. Use OpenPose to classify image colors

B. Directly print the image without loading any model

C. Load the OpenPose model, process the image, then extract keypoints

D. Skip model loading and only display raw pixels

Solution

1. Step 1: Recall OpenPose usage steps

   OpenPose requires loading its model, processing images, and extracting keypoints.

2. Step 2: Eliminate incorrect options

   Options B, C, and D ignore model loading or misuse OpenPose for unrelated tasks.

3. Final Answer:

   Load the OpenPose model, process the image, then extract keypoints -> Option C

4. Quick Check:

   Model load + process + keypoints = correct usage [OK]

Hint: Always load model before processing images [OK]

Common Mistakes:

• Skipping model loading step
• Using OpenPose for color classification
• Trying to process images without model

3. Given this Python snippet using OpenPose:

keypoints = openpose.process(image)
print(len(keypoints))

What does len(keypoints) represent?

medium

A. The number of detected people in the image

B. The number of pixels in the image

C. The number of colors detected

D. The number of image channels

Solution

1. Step 1: Understand what keypoints hold

   OpenPose returns keypoints for each detected person; length equals number of people detected.

2. Step 2: Compare with other options

   Pixels, colors, and channels are unrelated to keypoints length.

3. Final Answer:

   The number of detected people in the image -> Option A

4. Quick Check:

   len(keypoints) = people count [OK]

Hint: Keypoints list length = people detected [OK]

Common Mistakes:

• Thinking length is pixel count
• Confusing keypoints with colors
• Assuming length is image channels

4. You run this code snippet but get an error:

keypoints = openpose.process(image)
print(keypoints.shape)

What is the likely cause?

medium

A. keypoints is a list, not a numpy array, so it has no shape attribute

B. The image variable is not defined

C. OpenPose model was not loaded

D. print() function is used incorrectly

Solution

1. Step 1: Identify error cause

   keypoints from OpenPose is usually a list, which does not have a .shape attribute.

2. Step 2: Check other options

   Image undefined or model not loaded would cause different errors; print() usage is correct.

3. Final Answer:

   keypoints is a list, not a numpy array, so it has no shape attribute -> Option A

4. Quick Check:

   List has no .shape attribute [OK]

Hint: Use len() for lists, not .shape [OK]

Common Mistakes:

• Assuming keypoints is a numpy array
• Ignoring variable definition errors
• Blaming print() function

5. You want to use OpenPose to analyze a video with multiple people moving. Which approach is best to get accurate pose tracking over time?

hard

A. Process only the first frame and reuse keypoints for all frames

B. Process frames randomly without linking keypoints

C. Skip OpenPose and use color detection instead

D. Process each video frame with OpenPose and link detected keypoints across frames

Solution

1. Step 1: Understand video pose tracking

   Accurate tracking requires processing each frame and linking poses over time.

2. Step 2: Evaluate other options

   Reusing first frame keypoints ignores movement; color detection is unrelated; random processing loses continuity.

3. Final Answer:

   Process each video frame with OpenPose and link detected keypoints across frames -> Option D

4. Quick Check:

   Frame-by-frame + linking = accurate tracking [OK]

Hint: Track poses frame-by-frame, link keypoints [OK]

Common Mistakes:

• Using only first frame keypoints
• Confusing color detection with pose tracking
• Ignoring temporal linking of poses