Video writing in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Video writing

Which metric matters for Video Writing and WHY

When working with video writing in machine learning, the key metrics focus on how well the video frames are processed and saved without errors or loss. Important metrics include frame rate consistency, frame loss rate, and output video quality measures like PSNR (Peak Signal-to-Noise Ratio) or SSIM (Structural Similarity Index). These metrics matter because they tell us if the video writing process preserves the original video quality and timing, which is crucial for applications like video editing, surveillance, or autonomous driving.

Confusion Matrix or Equivalent Visualization

Video writing is not a classification task, so a confusion matrix does not apply directly. Instead, we use a frame-level error summary like this:

Frames Total: 1000
Frames Written Correctly: 995
Frames Lost or Corrupted: 5

Frame Loss Rate = 5 / 1000 = 0.005 (0.5%)

Quality Metrics:
PSNR: 38 dB (higher is better)
SSIM: 0.95 (scale 0 to 1, higher is better)

This shows how many frames were correctly saved and the quality of the saved video compared to the original.

Tradeoff: Frame Rate vs Quality

When writing video, there is often a tradeoff between speed (frame rate) and quality. Writing video faster may cause frame drops or compression artifacts, lowering quality. Writing slower can keep quality high but may not meet real-time needs.

For example:

• High frame rate (30 fps) with low compression: fewer frame drops, good quality, but larger file size and slower writing.
• Lower frame rate (15 fps) with high compression: faster writing, smaller files, but more frame loss and lower quality.

Choosing the right balance depends on the application. Real-time video calls need speed, while movie editing needs quality.

What "Good" vs "Bad" Metric Values Look Like

Good video writing metrics:

• Frame loss rate < 1%
• PSNR > 35 dB (close to original quality)
• SSIM > 0.9 (high structural similarity)
• Consistent frame rate matching input

Bad video writing metrics:

• Frame loss rate > 5%
• PSNR < 30 dB (noticeable quality loss)
• SSIM < 0.8 (visible distortions)
• Irregular frame rate causing jitter

Common Pitfalls in Video Writing Metrics

• Ignoring frame loss: Counting only file size or duration can hide dropped frames.
• Overlooking quality metrics: A video file may save quickly but with poor visual quality.
• Data leakage: Using original frames to evaluate quality without considering compression artifacts.
• Overfitting to speed: Optimizing only for fast writing can cause unstable frame rates and corrupted frames.

Self Check

Your video writing model saves video with 98% frame accuracy but the PSNR is 25 dB. Is it good?

Answer: No, because although most frames are saved, the PSNR of 25 dB indicates poor video quality with visible distortions. This means the video writing process is losing quality even if frames are mostly present. You should improve compression or encoding settings to raise quality.

Key Result

For video writing, low frame loss and high quality metrics like PSNR and SSIM are key to ensure good output.

Practice

1. What is the main purpose of using cv2.VideoWriter in computer vision?

easy

A. To save a series of images as a video file

B. To read video files frame by frame

C. To apply filters on images

D. To display images on screen

Solution

1. Step 1: Understand the function of cv2.VideoWriter

   cv2.VideoWriter is designed to save multiple images as a continuous video file.

2. Step 2: Differentiate from other OpenCV functions

   Functions like cv2.VideoCapture read videos, while VideoWriter writes videos.

3. Final Answer:

   To save a series of images as a video file -> Option A

4. Quick Check:

   Video writing = save images as video [OK]

Hint: VideoWriter saves frames into video files [OK]

Common Mistakes:

• Confusing VideoWriter with VideoCapture
• Thinking VideoWriter displays images
• Assuming VideoWriter applies image filters

2. Which of the following is the correct way to create a cv2.VideoWriter object for a 640x480 video at 30 fps using the 'XVID' codec?

easy

A. cv2.VideoWriter('output.avi', cv2.VideoWriter_fourcc(*'XVID'), 30, (640, 480))

B. cv2.VideoWriter('output.avi', 'XVID', 30, (640, 480))

C. cv2.VideoWriter('output.avi', cv2.VideoCapture_fourcc('XVID'), 30, (640, 480))

D. cv2.VideoWriter('output.avi', cv2.VideoWriter_fourcc('X', 'V', 'I', 'D'), 30, 640, 480)

Solution

1. Step 1: Use cv2.VideoWriter_fourcc correctly

   The codec must be passed as cv2.VideoWriter_fourcc(*'XVID') to get the fourcc code.

2. Step 2: Check the parameters order and types

   Parameters are filename, fourcc, fps, and frame size as a tuple (width, height).

3. Final Answer:

   cv2.VideoWriter('output.avi', cv2.VideoWriter_fourcc(*'XVID'), 30, (640, 480)) -> Option A

4. Quick Check:

   Correct codec and frame size syntax = cv2.VideoWriter('output.avi', cv2.VideoWriter_fourcc(*'XVID'), 30, (640, 480)) [OK]

Hint: Use cv2.VideoWriter_fourcc(*'XVID') for codec [OK]

Common Mistakes:

• Passing codec as string directly
• Using cv2.VideoCapture_fourcc instead of VideoWriter_fourcc
• Incorrect frame size format (not a tuple)

3. What will be the output of this code snippet?

import cv2
fourcc = cv2.VideoWriter_fourcc(*'MJPG')
out = cv2.VideoWriter('test.avi', fourcc, 20.0, (320,240))
frame = 255 * np.ones((240,320,3), dtype=np.uint8)
out.write(frame)
out.release()
print('Video saved')

medium

A. Nothing happens, no file is created

B. A video file named 'test.avi' with a white frame is saved and 'Video saved' is printed

C. RuntimeError because frame size does not match

D. NameError due to missing import of numpy

Solution

1. Step 1: Check imports

   The code uses np.ones but numpy (np) is not imported, causing a NameError.

2. Step 2: Understand error impact

   Without importing numpy, the code will fail before writing or saving video.

3. Final Answer:

   NameError due to missing import of numpy -> Option D

4. Quick Check:

   Missing numpy import = NameError [OK]

Hint: Always import numpy when using np [OK]

Common Mistakes:

• Assuming code runs without numpy import
• Ignoring frame size correctness
• Thinking print means success

4. Identify the error in this video writing code and select the fix:

import cv2
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('output.avi', fourcc, 25, (640, 480))
frame = cv2.imread('image.jpg')
out.write(frame)
out.release()

medium

A. Fix: Change codec to 'MJPG'

B. Fix: Resize frame to (640, 480) before writing

C. Fix: Use grayscale image instead of color

D. Fix: Increase fps to 30

Solution

1. Step 1: Check frame size vs VideoWriter size

   VideoWriter expects frames of size (640, 480). If frame size differs, write() fails or produces errors.

2. Step 2: Correct frame size by resizing

   Resize the loaded image to (640, 480) before writing to match VideoWriter settings.

3. Final Answer:

   Fix: Resize frame to (640, 480) before writing -> Option B

4. Quick Check:

   Frame size must match VideoWriter size [OK]

Hint: Match frame size to VideoWriter size [OK]

Common Mistakes:

• Ignoring frame size mismatch
• Changing codec unnecessarily
• Changing fps unrelated to error

5. You want to create a video from a list of images with different sizes. Which approach correctly handles this situation?

hard

A. Use VideoWriter without resizing; it auto-adjusts frame size

B. Write images directly without VideoWriter; just save as separate files

C. Resize all images to the same size before writing to VideoWriter

D. Change codec to 'X264' to support variable frame sizes

Solution

1. Step 1: Understand VideoWriter frame size requirement

   VideoWriter requires all frames to have the same size as specified at creation.

2. Step 2: Handle varying image sizes

   Resize all images to a consistent size before writing to avoid errors and ensure smooth video.

3. Final Answer:

   Resize all images to the same size before writing to VideoWriter -> Option C

4. Quick Check:

   Consistent frame size needed for video writing [OK]

Hint: Resize images to uniform size before video writing [OK]

Common Mistakes:

• Assuming VideoWriter auto-resizes frames
• Trying to write images without VideoWriter
• Changing codec to fix size issues