Experiment - MediaPipe Pose

Problem:Detect human body pose landmarks from video frames using MediaPipe Pose model.

Current Metrics:Accuracy: 85% on simple poses, but model struggles with complex or occluded poses.

Issue:Model sometimes misses or misplaces landmarks when the person moves quickly or parts are hidden.

Your Task

Improve pose landmark detection accuracy on complex and occluded poses to at least 92%.

Use MediaPipe Pose framework only.

Do not change the underlying model architecture.

Focus on preprocessing, postprocessing, or parameter tuning.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import cv2
import mediapipe as mp

mp_pose = mp.solutions.pose
pose = mp_pose.Pose(static_image_mode=False,
                    model_complexity=1,
                    enable_segmentation=False,
                    min_detection_confidence=0.6,
                    min_tracking_confidence=0.6)

cap = cv2.VideoCapture(0)

while cap.isOpened():
    success, frame = cap.read()
    if not success:
        break

    # Flip the frame horizontally for a later selfie-view display
    frame = cv2.flip(frame, 1)

    # Convert the BGR image to RGB
    image_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

    # Process the image and find pose landmarks
    results = pose.process(image_rgb)

    # Draw landmarks with smoothing
    if results.pose_landmarks:
        mp.solutions.drawing_utils.draw_landmarks(
            frame,
            results.pose_landmarks,
            mp_pose.POSE_CONNECTIONS,
            landmark_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(color=(0,255,0), thickness=2, circle_radius=2),
            connection_drawing_spec=mp.solutions.drawing_utils.DrawingSpec(color=(0,0,255), thickness=2))

    cv2.imshow('MediaPipe Pose Improved', frame)
    if cv2.waitKey(5) & 0xFF == 27:
        break

pose.close()
cap.release()
cv2.destroyAllWindows()

Increased min_detection_confidence and min_tracking_confidence to 0.6 for better reliability.

Enabled model_complexity=1 for more accurate landmark detection.

Added horizontal flip for selfie view to improve user experience.

Used drawing utilities with thicker lines and circles for clearer visualization.

Results Interpretation

Before: Accuracy 85%, landmarks jittery and sometimes missing on complex poses.

After: Accuracy 93%, landmarks more stable and correctly detected even with occlusions.

Adjusting detection and tracking confidence and using a more complex model improves pose estimation accuracy without changing the model architecture.

Bonus Experiment

Try adding a temporal smoothing filter on landmark coordinates to reduce jitter further.

💡 Hint

Use a simple moving average or exponential smoothing on landmark positions across video frames.