Action recognition basics in Computer Vision - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to load video frames for action recognition.

Computer Vision

import cv2

cap = cv2.VideoCapture('video.mp4')
ret, frame = cap.[1]()
if ret:
    print('Frame loaded')
cap.release()

Drag options to blanks, or click blank then click option'

Acapture

Bread

Cload

Dget

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'get' instead of 'read' causes an error.

Using 'load' is not a valid method.

The read() method reads the next video frame from the capture object.

2fill in blank

medium

Complete the code to extract features from frames using a pretrained CNN model.

Computer Vision

from torchvision import models, transforms
import torch

model = models.resnet18(pretrained=True)
model.eval()

preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])

input_tensor = preprocess(frame)
input_batch = input_tensor.unsqueeze(0)

with torch.no_grad():
    features = model.[1](input_batch)

Drag options to blanks, or click blank then click option'

Aforward

Bpredict

Ctransform

Dfit

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'predict' causes attribute error.

Using 'fit' is for training, not inference.

The forward method runs the input through the model to get output features.

3fill in blank

hard

Fix the error in the code to correctly compute accuracy for action recognition predictions.

Computer Vision

correct = 0
for pred, label in zip(predictions, labels):
    if pred == [1]:
        correct += 1
accuracy = correct / len(labels)
print(f'Accuracy: {accuracy:.2f}')

Drag options to blanks, or click blank then click option'

Apred

Bpredictions

Clabels

Dlabel

Attempts:

3 left

💡 Hint

Common Mistakes

Comparing prediction to predictions list causes wrong results.

Comparing to 'pred' always true, inflating accuracy.

We compare each predicted label pred to the true label label to count correct predictions.

4fill in blank

hard

Fill both blanks to create a dictionary of frame indices and their corresponding action labels.

Computer Vision

frame_labels = {i: [1] for i, [2] in enumerate(predicted_actions)}

Drag options to blanks, or click blank then click option'

Alabel

Baction

Cpredicted_actions

Dlabel_idx

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'predicted_actions' as loop variable causes error.

Using 'label_idx' is not defined in loop.

We use action as the value and action as the loop variable for predicted actions.

5fill in blank

hard

Fill all three blanks to filter frames with confidence above threshold and create a result dictionary.

Computer Vision

result = {frame_id: [1] for frame_id, (label, conf) in frame_data.items() if conf [2] [3]

Drag options to blanks, or click blank then click option'

Alabel

B>

C0.8

Dconf

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'conf' as value instead of label.

Using '<' instead of '>' changes filtering logic.

We keep labels where confidence is greater than 0.8.

Practice

1. What is the main goal of action recognition in computer vision?

easy

A. To generate captions for images

B. To detect objects in images

C. To enhance image resolution

D. To identify human movements in videos

Solution

1. Step 1: Understand the purpose of action recognition

   Action recognition focuses on understanding what actions or movements humans perform in videos.

2. Step 2: Compare with other tasks

   Detecting objects, generating captions, or enhancing resolution are different tasks unrelated to recognizing actions.

3. Final Answer:

   To identify human movements in videos -> Option D

4. Quick Check:

   Action recognition = Identify human movements [OK]

Hint: Action recognition = understanding human movements in videos [OK]

Common Mistakes:

• Confusing action recognition with object detection
• Thinking it generates image captions
• Assuming it improves image quality

2. Which of the following is the correct way to represent a video input for an action recognition model?

easy

A. A sequence of image frames

B. A single grayscale image

C. A text description of the action

D. A 1D audio signal

Solution

1. Step 1: Identify video data format

   Videos are made of many image frames shown in order, so a sequence of frames is the correct input.

2. Step 2: Eliminate incorrect options

   A single image or text or audio does not represent the full video needed for action recognition.

3. Final Answer:

   A sequence of image frames -> Option A

4. Quick Check:

   Video input = sequence of frames [OK]

Hint: Videos = many frames in order, not single images [OK]

Common Mistakes:

• Using a single image instead of multiple frames
• Confusing video input with text or audio
• Ignoring the temporal sequence of frames

3. Consider this Python snippet for extracting features from video frames for action recognition:

features = []
for frame in video_frames:
    feat = extract_features(frame)
    features.append(feat)
print(len(features))

If video_frames contains 10 frames, what will be the output?

medium

A. 10

B. 9

C. 0

D. Error

Solution

1. Step 1: Understand the loop over frames

   The loop runs once for each frame in video_frames, which has 10 frames.

2. Step 2: Count how many features are appended

   Each iteration appends one feature, so after 10 iterations, features has length 10.

3. Final Answer:

   10 -> Option A

4. Quick Check:

   Number of frames = features length = 10 [OK]

Hint: One feature per frame means length equals number of frames [OK]

Common Mistakes:

• Off-by-one errors counting features
• Assuming extract_features returns multiple items
• Thinking the list is empty before print

4. You have this code snippet for action recognition training:

for video, label in dataset:
    features = extract_features(video)
    prediction = model.predict(features)
    loss = loss_function(prediction, label)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

The training loss does not decrease after many epochs. What is a likely error?

medium

A. Optimizer step is missing

B. Loss function is not called

C. Features are extracted frame-by-frame but model expects video clips

D. Labels are not used in prediction

Solution

1. Step 1: Analyze feature extraction and model input

   If features are extracted frame-by-frame but the model expects a clip (multiple frames together), the input shape mismatch can cause poor learning.

2. Step 2: Check other training steps

   Loss function is called, optimizer steps are present, and labels are used in loss, so these are correct.

3. Final Answer:

   Features are extracted frame-by-frame but model expects video clips -> Option C

4. Quick Check:

   Input shape mismatch = training loss stuck [OK]

Hint: Check if model input matches feature extraction format [OK]

Common Mistakes:

• Ignoring input shape mismatch
• Assuming loss or optimizer calls are missing
• Not verifying label usage in loss

5. You want to improve an action recognition model that uses only spatial features from single frames. Which approach is best to capture motion information?

hard

A. Train on grayscale frames instead of color

B. Use 3D convolutional neural networks on video clips

C. Add dropout layers to the model

D. Increase image resolution of single frames

Solution

1. Step 1: Understand spatial vs temporal features

   Spatial features come from single frames; motion requires temporal features across frames.

2. Step 2: Identify model type capturing motion

   3D CNNs process multiple frames together, capturing motion and temporal info effectively.

3. Step 3: Evaluate other options

   Increasing resolution, dropout, or grayscale do not add motion info.

4. Final Answer:

   Use 3D convolutional neural networks on video clips -> Option B

5. Quick Check:

   3D CNNs capture motion = better action recognition [OK]

Hint: Motion needs temporal models like 3D CNNs, not just images [OK]

Common Mistakes:

• Thinking higher resolution adds motion info
• Confusing regular CNNs with 3D CNNs
• Ignoring temporal dimension in videos