Homography and image alignment in Computer Vision

Introduction

Homography helps us match one picture to another by finding how one image can be shifted, rotated, or stretched to fit the other. This is useful to line up images taken from different angles.

When you want to stitch photos together to make a panorama.

When you want to overlay graphics on a video or image correctly.

When you want to correct the perspective of a photo taken at an angle.

When you want to track objects moving between camera views.

When you want to align images for comparison or analysis.

Syntax

Computer Vision

import cv2
import numpy as np

# Find homography matrix
homography_matrix, status = cv2.findHomography(src_points, dst_points, method=cv2.RANSAC)

# Warp source image to align with destination image
aligned_image = cv2.warpPerspective(source_image, homography_matrix, (width, height))

src_points and dst_points are arrays of matching points between two images.

cv2.findHomography uses RANSAC to ignore bad matches and find the best transformation.

Examples

Basic example with four matching points to find homography.

Computer Vision

src_points = np.array([[10, 20], [50, 80], [90, 40], [120, 100]])
dst_points = np.array([[15, 25], [55, 85], [95, 45], [125, 105]])
homography_matrix, status = cv2.findHomography(src_points, dst_points, cv2.RANSAC)

Edge case: empty points will cause failure; need at least 4 points.

Computer Vision

src_points = np.array([])
dst_points = np.array([])
homography_matrix, status = cv2.findHomography(src_points, dst_points, cv2.RANSAC)

Edge case: only one point is not enough to compute homography.

Computer Vision

src_points = np.array([[10, 20]])
dst_points = np.array([[15, 25]])
homography_matrix, status = cv2.findHomography(src_points, dst_points, cv2.RANSAC)

Sample Model

This program creates a simple square image, rotates it to make a destination image, finds the homography, aligns the source to destination, and checks how close they are.

Computer Vision

import cv2
import numpy as np

# Create two simple images with a square shifted
source_image = np.zeros((200, 200), dtype=np.uint8)
cv2.rectangle(source_image, (50, 50), (150, 150), 255, -1)  # white square

# Destination image has the square shifted and rotated
angle = 10  # degrees
M = cv2.getRotationMatrix2D((100, 100), angle, 1)
destination_image = cv2.warpAffine(source_image, M, (200, 200))

# Define matching points manually (corners of the square)
src_points = np.array([[50, 50], [150, 50], [150, 150], [50, 150]], dtype=np.float32)
dst_points = cv2.transform(np.array([src_points]), M)[0]

# Find homography matrix
homography_matrix, status = cv2.findHomography(src_points, dst_points, cv2.RANSAC)

# Warp source image to align with destination
aligned_image = cv2.warpPerspective(source_image, homography_matrix, (200, 200))

# Check difference between aligned image and destination
difference = cv2.absdiff(aligned_image, destination_image)

# Print homography matrix and sum of difference pixels
print("Homography matrix:")
print(homography_matrix)
print(f"Sum of absolute difference pixels: {np.sum(difference)}")

Important Notes

Finding homography requires at least 4 pairs of matching points.

The time complexity depends on the number of points and RANSAC iterations, usually fast for small sets.

Common mistake: Using points that do not correspond well causes wrong homography.

Use homography when images are related by perspective transform; use simpler transforms if only translation or rotation.

Summary

Homography finds how to map one image to another using matching points.

It helps align images taken from different views or angles.

Use cv2.findHomography and cv2.warpPerspective to compute and apply homography.

Practice

1. What is the main purpose of computing a homography matrix in image alignment?

easy

A. To increase the brightness of an image

B. To detect edges in an image

C. To segment objects in an image

D. To find a transformation that maps points from one image to another

Solution

1. Step 1: Understand homography concept

   Homography is a matrix that relates points between two images taken from different views.

2. Step 2: Identify its use in image alignment

   It helps to map points from one image to corresponding points in another to align them.

3. Final Answer:

   To find a transformation that maps points from one image to another -> Option D

4. Quick Check:

   Homography = Point mapping [OK]

Hint: Homography maps points between images [OK]

Common Mistakes:

• Confusing homography with edge detection
• Thinking homography changes image brightness
• Mixing homography with image segmentation

2. Which OpenCV function is used to compute the homography matrix from matched points?

easy

A. cv2.warpPerspective()

B. cv2.findHomography()

C. cv2.matchTemplate()

D. cv2.resize()

Solution

1. Step 1: Identify function for homography calculation

   cv2.findHomography() computes the homography matrix from matched point sets.

2. Step 2: Differentiate from other functions

   cv2.warpPerspective applies the homography, matchTemplate finds template matches, resize changes image size.

3. Final Answer:

   cv2.findHomography() -> Option B

4. Quick Check:

   Compute homography = findHomography [OK]

Hint: Find homography matrix with cv2.findHomography() [OK]

Common Mistakes:

• Using warpPerspective to compute homography
• Confusing template matching with homography calculation
• Trying to resize image to get homography

3. Given the following code snippet, what will be the shape of aligned_img after applying homography?

import cv2
import numpy as np
img1 = cv2.imread('img1.jpg')
img2 = cv2.imread('img2.jpg')
pts1 = np.array([[10,10],[100,10],[100,100],[10,100]])
pts2 = np.array([[12,14],[102,12],[98,110],[14,108]])
H, _ = cv2.findHomography(pts1, pts2)
aligned_img = cv2.warpPerspective(img1, H, (img2.shape[1], img2.shape[0]))

medium

A. Same height and width as img1

B. Shape is (4, 2) because of points

C. Same height and width as img2

D. Shape depends on H matrix size

Solution

1. Step 1: Understand warpPerspective parameters

   The third parameter in warpPerspective sets output image size as (width, height).

2. Step 2: Check given size argument

   It uses (img2.shape[1], img2.shape[0]) which is width and height of img2.

3. Final Answer:

   Same height and width as img2 -> Option C

4. Quick Check:

   Output size = img2.shape [OK]

Hint: warpPerspective size param sets output image shape [OK]

Common Mistakes:

• Assuming output shape matches img1
• Thinking homography matrix size affects output shape
• Confusing point arrays with image shape

4. You wrote this code to align two images but get a distorted output. What is the likely error?

H, status = cv2.findHomography(pts1, pts2)
aligned = cv2.warpPerspective(img1, pts1, (img2.shape[1], img2.shape[0]))

medium

A. Using pts1 instead of H in warpPerspective

B. Swapping pts1 and pts2 in findHomography

C. Not converting points to float32

D. Missing cv2.imshow to display image

Solution

1. Step 1: Check warpPerspective arguments

   warpPerspective expects the homography matrix as the second argument, not point arrays.

2. Step 2: Identify incorrect argument usage

   Code passes pts1 (points) instead of H (homography matrix), causing distortion.

3. Final Answer:

   Using pts1 instead of H in warpPerspective -> Option A

4. Quick Check:

   warpPerspective needs homography matrix [OK]

Hint: Pass homography matrix, not points, to warpPerspective [OK]

Common Mistakes:

• Passing points instead of homography matrix
• Swapping source and destination points in findHomography
• Ignoring data type requirements for points

5. You want to stitch two images taken from different angles into a panorama. Which sequence of steps correctly uses homography for alignment?

hard

A. Detect keypoints -> Match points -> Compute homography -> Warp one image -> Blend images

B. Resize images -> Compute homography -> Detect edges -> Warp images -> Blend images

C. Match points -> Resize images -> Compute homography -> Warp images -> Detect keypoints

D. Warp images -> Detect keypoints -> Compute homography -> Match points -> Blend images

Solution

1. Step 1: Detect and match keypoints

   First, find keypoints in both images and match them to get corresponding points.

2. Step 2: Compute homography and warp image

   Use matched points to compute homography, then warp one image to align with the other.

3. Step 3: Blend images to create panorama

   Finally, blend the aligned images smoothly to form a panorama.

4. Final Answer:

   Detect keypoints -> Match points -> Compute homography -> Warp one image -> Blend images -> Option A

5. Quick Check:

   Correct panorama steps = Detect keypoints -> Match points -> Compute homography -> Warp one image -> Blend images [OK]

Hint: Detect -> Match -> Compute -> Warp -> Blend for panorama [OK]

Common Mistakes:

• Resizing images before matching points
• Warping images before computing homography
• Detecting keypoints after warping images