Feature extraction approach in TensorFlow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main goal of feature extraction in machine learning?

The main goal of feature extraction is to transform raw data into a set of meaningful features that make it easier for a model to learn patterns and make predictions.

intermediate

How does feature extraction differ from feature selection?

Feature extraction creates new features by transforming the original data, while feature selection chooses a subset of existing features without changing them.

intermediate

In TensorFlow, what is a common way to perform feature extraction using a pre-trained model?

A common way is to use a pre-trained model like MobileNet or ResNet, remove its final classification layer, and use the output of the last layer before classification as extracted features.

beginner

Why is feature extraction useful when working with images?

Feature extraction helps convert complex image data into simpler, informative features that capture important patterns like edges or shapes, making it easier for models to learn and predict.

intermediate

What is transfer learning and how is it related to feature extraction?

Transfer learning uses a pre-trained model on a new task. Feature extraction is part of this process where the pre-trained model's learned features are reused to help learn the new task faster and with less data.

What does feature extraction do in a machine learning pipeline?

A Trains the model directly

B Removes irrelevant features

C Transforms raw data into useful features

D Splits data into training and testing sets

Feature extraction transforms raw data into useful features that help the model learn better.

Which TensorFlow method is commonly used to load a pre-trained model for feature extraction?

A tf.image.resize

B tf.keras.applications.MobileNetV2

C tf.keras.layers.Dense

D tf.data.Dataset.from_tensor_slices

tf.keras.applications.MobileNetV2 loads a pre-trained model often used for feature extraction.

In feature extraction, what part of a pre-trained model is usually removed?

A The final classification layer

B The input layer

C The first convolutional layer

D The optimizer

The final classification layer is removed to use the model's learned features without its original output.

Why might feature extraction reduce training time?

A Because it uses pre-learned features instead of learning from scratch

B Because it uses fewer data points

C Because it removes all features

D Because it trains on raw data

Feature extraction uses pre-learned features, so the model doesn't need to learn everything from scratch, saving time.

Which of these is NOT a benefit of feature extraction?

A Improves model accuracy

B Simplifies complex data

C Reduces training time

D Automatically labels data

Feature extraction does not label data; it transforms data into useful features.

Explain how feature extraction works using a pre-trained TensorFlow model for image data.

Think about how a model trained on many images can help with new image tasks.

Describe the difference between feature extraction and feature selection in simple terms.

One changes data, the other chooses from data.

Practice

1. What is the main purpose of using feature extraction with a pre-trained TensorFlow model?

easy

A. To use learned features from a large dataset to help a new task

B. To train a model from scratch with random weights

C. To increase the size of the dataset automatically

D. To reduce the number of layers in the model

Solution

1. Step 1: Understand feature extraction concept

   Feature extraction uses a pre-trained model to get useful features learned from a large dataset.

2. Step 2: Identify the benefit for new tasks

   This helps when you have little data or want faster training by reusing learned features.

3. Final Answer:

   To use learned features from a large dataset to help a new task -> Option A

4. Quick Check:

   Feature extraction = reuse learned features [OK]

Hint: Feature extraction reuses knowledge from big datasets [OK]

Common Mistakes:

• Thinking feature extraction trains from scratch
• Confusing feature extraction with data augmentation
• Believing it reduces model layers automatically

2. Which TensorFlow code snippet correctly freezes the base model layers for feature extraction?

easy

A. base_model.trainable = True

B. base_model.trainable = False

C. base_model.freeze()

D. base_model.layers.freeze = True

Solution

1. Step 1: Recall how to freeze layers in TensorFlow

   Setting trainable = False on the base model freezes its weights.

2. Step 2: Check each option's correctness

   base_model.trainable = False correctly sets base_model.trainable = False. Others are invalid syntax or do not freeze layers.

3. Final Answer:

   base_model.trainable = False -> Option B

4. Quick Check:

   Freeze layers = trainable False [OK]

Hint: Freeze layers by setting trainable to False [OK]

Common Mistakes:

• Using trainable = True to freeze layers
• Calling non-existent freeze() method
• Trying to set freeze attribute on layers directly

3. Given this code, what is the output shape of features?

base_model = tf.keras.applications.MobileNetV2(input_shape=(128,128,3), include_top=False)
base_model.trainable = False
inputs = tf.keras.Input(shape=(128,128,3))
features = base_model(inputs)
print(features.shape)

medium

A. (None, 4, 4, 1280)

B. (None, 7, 7, 512)

C. (None, 1, 1, 1000)

D. (None, 128, 128, 3)

Solution

1. Step 1: Understand MobileNetV2 output with include_top=False

   MobileNetV2 outputs feature maps of size (4,4,1280) for 128x128 input when top layers are excluded.

2. Step 2: Confirm batch dimension is None

   The batch size is None (unspecified), so output shape is (None, 4, 4, 1280).

3. Final Answer:

   (None, 4, 4, 1280) -> Option A

4. Quick Check:

   MobileNetV2 feature shape = (None,4,4,1280) [OK]

Hint: MobileNetV2 outputs 4x4x1280 features for 128x128 input [OK]

Common Mistakes:

• Confusing output with classification layer shape
• Assuming output shape equals input shape
• Ignoring batch dimension None

4. Identify the error in this feature extraction code snippet:

base_model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False)
base_model.trainable = False
inputs = tf.keras.Input(shape=(224,224,3))
x = base_model(inputs)
x = tf.keras.layers.Dense(10)(x)
model = tf.keras.Model(inputs, x)

medium

A. Base model weights not loaded correctly

B. Input shape is incorrect for ResNet50

C. Dense layer applied to 4D tensor without flattening

D. trainable should be True to freeze layers

Solution

1. Step 1: Check tensor shapes before Dense layer

   Base model outputs 4D tensor (batch, height, width, channels). Dense expects 2D input.

2. Step 2: Identify missing flatten or global pooling

   Applying Dense directly to 4D tensor causes error; flatten or global pooling needed first.

3. Final Answer:

   Dense layer applied to 4D tensor without flattening -> Option C

4. Quick Check:

   Dense needs 2D input, flatten first [OK]

Hint: Flatten or pool before Dense layer [OK]

Common Mistakes:

• Applying Dense directly to 4D output
• Confusing input shape requirements
• Misunderstanding trainable flag usage

5. You want to use feature extraction with MobileNetV2 on 64x64 images but get an error when adding a Dense layer after the base model. What is the best way to fix this?

hard

A. Set base_model.trainable = True to allow training

B. Change input shape to 224x224 to match MobileNetV2 default

C. Remove the Dense layer and use base model output directly

D. Add a GlobalAveragePooling2D layer before the Dense layer

Solution

1. Step 1: Understand why Dense layer causes error

   MobileNetV2 outputs 4D feature maps; Dense layers need 2D input.

2. Step 2: Use GlobalAveragePooling2D to convert 4D to 2D

   Adding GlobalAveragePooling2D flattens spatial dimensions, making output suitable for Dense.

3. Step 3: Evaluate other options

   Changing input size is not required; training base model is optional; removing Dense loses classification ability.

4. Final Answer:

   Add a GlobalAveragePooling2D layer before the Dense layer -> Option D

5. Quick Check:

   Pool features before Dense layer [OK]

Hint: Use pooling to flatten features before Dense [OK]

Common Mistakes:

• Changing input size unnecessarily
• Trying to train frozen base model
• Skipping pooling and causing shape errors