Hand and face landmark detection in Computer Vision

Introduction

Hand and face landmark detection helps computers find important points on your hands and face. This lets machines understand gestures and expressions like a friend would.

To control a game using hand gestures without touching anything.

To add fun face filters like sunglasses or hats in a video chat app.

To help robots recognize human emotions by reading facial expressions.

To count how many fingers are raised for a sign language app.

To track hand movements for virtual reality or fitness coaching.

Syntax

Computer Vision

import mediapipe as mp

mp_hands = mp.solutions.hands
mp_face_mesh = mp.solutions.face_mesh

with mp_hands.Hands() as hands, mp_face_mesh.FaceMesh() as face_mesh:
    results_hands = hands.process(image_rgb)
    results_face = face_mesh.process(image_rgb)

This example uses the MediaPipe library, which has ready-made models for hand and face landmarks.

You need to convert your image to RGB before processing because the models expect that format.

Examples

Detect hand landmarks in a single image.

Computer Vision

import mediapipe as mp
mp_hands = mp.solutions.hands
with mp_hands.Hands() as hands:
    results = hands.process(image_rgb)

Detect face landmarks in a single image.

Computer Vision

import mediapipe as mp
mp_face_mesh = mp.solutions.face_mesh
with mp_face_mesh.FaceMesh() as face_mesh:
    results = face_mesh.process(image_rgb)

Detect both hand and face landmarks in the same image.

Computer Vision

import mediapipe as mp
mp_hands = mp.solutions.hands
mp_face_mesh = mp.solutions.face_mesh
with mp_hands.Hands() as hands, mp_face_mesh.FaceMesh() as face_mesh:
    results_hands = hands.process(image_rgb)
    results_face = face_mesh.process(image_rgb)

Sample Model

This program loads an image, detects hand and face landmarks, and prints how many were found.

Computer Vision

import cv2
import mediapipe as mp

mp_hands = mp.solutions.hands
mp_face_mesh = mp.solutions.face_mesh
mp_drawing = mp.solutions.drawing_utils

# Load an example image
image = cv2.imread('hand_face.jpg')
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

with mp_hands.Hands(static_image_mode=True, max_num_hands=2) as hands, \
     mp_face_mesh.FaceMesh(static_image_mode=True) as face_mesh:
    results_hands = hands.process(image_rgb)
    results_face = face_mesh.process(image_rgb)

    # Print number of hands detected
    num_hands = len(results_hands.multi_hand_landmarks) if results_hands.multi_hand_landmarks else 0
    print(f'Hands detected: {num_hands}')

    # Print number of face landmarks detected
    num_face_landmarks = len(results_face.multi_face_landmarks[0].landmark) if results_face.multi_face_landmarks else 0
    print(f'Face landmarks detected: {num_face_landmarks}')

Important Notes

Make sure your input image is clear and well-lit for better detection.

MediaPipe returns landmarks as points with x, y, z coordinates normalized between 0 and 1.

You can draw landmarks on images using MediaPipe's drawing utilities for visualization.

Summary

Hand and face landmark detection finds key points on hands and faces in images or videos.

This helps computers understand gestures and expressions for many fun and useful apps.

MediaPipe is a popular tool that makes it easy to detect these landmarks with just a few lines of code.