Architecture search concepts in Computer Vision - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main goal of Neural Architecture Search (NAS)?

Neural Architecture Search (NAS) is a technique used in machine learning. What is its main goal?

A To increase the size of the training dataset by data augmentation

B To manually tune hyperparameters like learning rate and batch size

C To automatically find the best neural network design for a specific task

D To reduce the number of layers in a neural network to speed up training

💡 Hint

Think about what NAS tries to automate in model building.

NAS automates the process of designing neural network architectures to find the best structure for a given task, instead of manual trial and error.

❓ Model Choice

intermediate

Which architecture search method uses reinforcement learning?

Among the following architecture search methods, which one typically uses reinforcement learning to guide the search?

A Evolutionary algorithms

B Controller RNN-based search

C Random search

D Grid search

💡 Hint

Look for the method that uses a neural network to propose architectures.

Controller RNN-based NAS uses reinforcement learning where the controller RNN proposes architectures and learns from their performance.

❓ Hyperparameter

advanced

In architecture search, what does the 'search space' define?

When performing architecture search, what does the 'search space' specify?

A The range of possible neural network architectures to explore

B The hardware resources available for training

C The learning rate schedule during training

D The dataset used for training the model

💡 Hint

Think about what the search algorithm is choosing from.

The search space defines all possible architectures that the search method can consider during the search process.

❓ Metrics

advanced

Which metric is commonly used to evaluate candidate architectures during NAS?

During Neural Architecture Search, which metric is most commonly used to evaluate how good a candidate architecture is?

A Time taken to train the model

B Training loss on the full training set

C Number of parameters in the model

D Validation accuracy on a held-out dataset

💡 Hint

Think about what shows how well the model generalizes.

Validation accuracy measures how well the architecture performs on unseen data, which is key to selecting good models.

🔧 Debug

expert

What error occurs if the search space is too large without constraints?

In Neural Architecture Search, if the search space is extremely large and unconstrained, what is the most likely problem that will occur?

A The search will take too long and may never find a good architecture

B The model will overfit the training data immediately

C The training will fail due to incompatible layer sizes

D The search will produce architectures with zero parameters

💡 Hint

Consider the impact of a huge number of options to explore.

A very large search space makes the search computationally expensive and slow, often preventing finding good architectures in reasonable time.

Practice

1. What is the main goal of architecture search in computer vision models?

easy

A. To collect more training data

B. To manually tune model parameters

C. To automatically find the best model design

D. To reduce image resolution

Solution

1. Step 1: Understand architecture search purpose

   Architecture search aims to find the best model design automatically without manual trial and error.

2. Step 2: Compare options

   Options B, C, and D do not describe architecture search goals. Only To automatically find the best model design matches the goal.

3. Final Answer:

   To automatically find the best model design -> Option C

4. Quick Check:

   Architecture search = automatic best design [OK]

Hint: Architecture search = automatic model design finder [OK]

Common Mistakes:

• Confusing architecture search with data collection
• Thinking it manually tunes parameters
• Mixing it with image preprocessing

2. Which of the following is a correct way to describe a search space in architecture search?

easy

A. A set of possible model designs to explore

B. The training dataset used for the model

C. The final accuracy metric after training

D. The hardware used to run the model

Solution

1. Step 1: Define search space

   Search space is the collection of all possible model designs or configurations that the search will try.

2. Step 2: Eliminate incorrect options

   Options B, C, and D relate to data, metrics, or hardware, not the search space itself.

3. Final Answer:

   A set of possible model designs to explore -> Option A

4. Quick Check:

   Search space = possible designs [OK]

Hint: Search space = all model options to try [OK]

Common Mistakes:

• Confusing search space with dataset
• Thinking search space is a metric
• Mixing search space with hardware details

3. Consider this pseudocode for architecture search:

for model in search_space:
    accuracy = train_and_evaluate(model)
    if accuracy > best_accuracy:
        best_model = model
        best_accuracy = accuracy
print(best_accuracy)

What does this code output?

medium

A. The list of all models tested

B. The accuracy of the best model found

C. The training loss of the last model

D. The total number of models in search_space

Solution

1. Step 1: Analyze the loop

   The loop trains and evaluates each model, updating best_accuracy if current accuracy is higher.

2. Step 2: Understand the print statement

   After checking all models, it prints the highest accuracy found among them.

3. Final Answer:

   The accuracy of the best model found -> Option B

4. Quick Check:

   Prints best accuracy = highest accuracy [OK]

Hint: Code prints highest accuracy found during search [OK]

Common Mistakes:

• Thinking it prints number of models
• Confusing accuracy with loss
• Assuming it prints all models

4. The following code snippet is intended to find the best model architecture, but it has a bug:

best_accuracy = 0
for model in search_space:
    accuracy = train_and_evaluate(model)
    if accuracy < best_accuracy:
        best_model = model
        best_accuracy = accuracy
print(best_accuracy)

What is the bug?

medium

A. best_accuracy should start at 1 instead of 0

B. train_and_evaluate should return loss, not accuracy

C. The print statement should print best_model, not best_accuracy

D. The comparison operator should be > instead of <

Solution

1. Step 1: Understand the goal

   The goal is to find the model with the highest accuracy, so we want to update when accuracy is greater than best_accuracy.

2. Step 2: Identify the bug

   The code uses accuracy < best_accuracy, which updates for worse accuracy, so it should be accuracy > best_accuracy.

3. Final Answer:

   The comparison operator should be > instead of < -> Option D

4. Quick Check:

   Use > to find best accuracy [OK]

Hint: Best accuracy means use >, not < in comparison [OK]

Common Mistakes:

• Starting best_accuracy at wrong value
• Printing wrong variable
• Confusing accuracy with loss

5. You want to speed up architecture search by reducing the search space size. Which strategy is best?

hard

A. Limit model depth and number of layers to a smaller range

B. Increase the number of training epochs for each model

C. Use a slower but more accurate optimizer

D. Train all models on the full dataset without sampling

Solution

1. Step 1: Understand search space impact

   Reducing search space size means limiting the number of possible model designs to try.

2. Step 2: Evaluate options

   Limit model depth and number of layers to a smaller range reduces model complexity range, shrinking search space. Options A, B, and D increase training time or data size, slowing search.

3. Final Answer:

   Limit model depth and number of layers to a smaller range -> Option A

4. Quick Check:

   Smaller search space = fewer model options [OK]

Hint: Shrink search space by limiting model complexity [OK]

Common Mistakes:

• Thinking more training epochs speed up search
• Choosing slower optimizers to improve speed
• Using full dataset always speeds search