Feature engineering pipelines in MLOps - Step-by-Step Execution

Process Flow - Feature engineering pipelines

Raw Data Input

↓

Data Cleaning

↓

Feature Extraction

↓

Feature Transformation

↓

Feature Selection

↓

Output: Processed Features

↓

Model Training / Deployment

The pipeline starts with raw data, then cleans it, extracts and transforms features, selects the best ones, and outputs processed features for model use.

Execution Sample

MLOps

pipeline = Pipeline([
  ('clean', DataCleaner()),
  ('extract', FeatureExtractor()),
  ('transform', FeatureTransformer()),
  ('select', FeatureSelector())
])
processed_features = pipeline.fit_transform(raw_data)

This code builds a feature engineering pipeline that cleans, extracts, transforms, and selects features from raw data.

Process Table

Step	Pipeline Stage	Input Data State	Action	Output Data State
1	DataCleaner	Raw data with missing values and noise	Remove missing values and fix errors	Cleaned data without missing values
2	FeatureExtractor	Cleaned data	Extract relevant features (e.g., date parts, text tokens)	Data with new extracted features
3	FeatureTransformer	Extracted features	Scale, encode, or normalize features	Transformed features ready for selection
4	FeatureSelector	Transformed features	Select most important features	Final feature set for model
5	End	Final feature set	Pipeline complete	Processed features ready for training

💡 Pipeline finishes after feature selection producing processed features for model training.

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	Final
data_state	Raw data	Cleaned data	Data with extracted features	Transformed features	Selected features	Processed features

Key Moments - 3 Insights

Why do we need to clean data before extracting features?

Cleaning removes errors and missing values that could cause wrong feature extraction, as shown in step 1 where raw data becomes clean data before extraction.

What happens if we skip feature transformation?

Skipping transformation means features may not be scaled or encoded properly, causing poor model performance. Step 3 shows how transformation prepares features for selection.

Why is feature selection important at the end?

Feature selection reduces noise and improves model efficiency by keeping only important features, as step 4 outputs a smaller, relevant feature set.

Visual Quiz - 3 Questions

Look at the execution table, what is the data state after step 2?

A Raw data

B Cleaned data

C Data with extracted features

D Selected features

💡 Hint

Check the 'Output Data State' column for step 2 in the execution table.

At which step does the pipeline remove missing values?

A Step 3

B Step 1

C Step 4

D Step 2

💡 Hint

Look at the 'Action' column in the execution table for step 1.

If feature selection is skipped, how would the final data state change?

A It would be transformed features instead of selected features

B It would remain raw data

C It would be cleaned data

D It would be extracted features

💡 Hint

Compare the output states after step 3 and step 4 in the variable tracker.

Concept Snapshot

Feature engineering pipelines process raw data step-by-step:
1. Clean data to fix errors
2. Extract useful features
3. Transform features (scale, encode)
4. Select important features
This pipeline outputs ready-to-use features for model training.

Full Transcript

Feature engineering pipelines take raw data and process it through stages: cleaning, feature extraction, transformation, and selection. Each step changes the data state, improving quality and relevance. Cleaning removes errors, extraction creates new features, transformation prepares features for modeling, and selection picks the best features. This stepwise process ensures models get the best input data for training and deployment.

Practice

1. What is the main purpose of a feature engineering pipeline in MLOps?

easy

A. To automate and standardize data preparation steps

B. To deploy machine learning models to production

C. To monitor model performance after deployment

D. To collect raw data from external sources

Solution

1. Step 1: Understand the role of feature engineering pipelines

   Feature engineering pipelines automate the process of transforming raw data into features for model training and testing.

2. Step 2: Differentiate from other MLOps tasks

   Deploying models, monitoring, and data collection are separate tasks from feature engineering pipelines.

3. Final Answer:

   To automate and standardize data preparation steps -> Option A

4. Quick Check:

   Feature engineering pipeline = automate data prep [OK]

Hint: Feature pipelines automate data prep, not deployment or monitoring [OK]

Common Mistakes:

• Confusing feature pipelines with model deployment
• Thinking pipelines collect raw data
• Mixing up monitoring with feature engineering

2. Which of the following is the correct way to define a simple feature engineering pipeline step using scikit-learn's Pipeline?

easy

A. pipeline = Pipeline([('scaler', StandardScaler()), ('pca', PCA(n_components=2))])

B. pipeline = Pipeline('scaler', StandardScaler(), 'pca', PCA(n_components=2))

C. pipeline = Pipeline({'scaler': StandardScaler(), 'pca': PCA(n_components=2)})

D. pipeline = Pipeline(StandardScaler(), PCA(n_components=2))

Solution

1. Step 1: Recall scikit-learn Pipeline syntax

   Pipeline expects a list of tuples, each tuple with a name and a transformer object.

2. Step 2: Check each option's syntax

   pipeline = Pipeline([('scaler', StandardScaler()), ('pca', PCA(n_components=2))]) correctly uses a list of tuples. Options B, C, and D use incorrect argument formats.

3. Final Answer:

   pipeline = Pipeline([('scaler', StandardScaler()), ('pca', PCA(n_components=2))]) -> Option A

4. Quick Check:

   Pipeline needs list of (name, transformer) tuples [OK]

Hint: Pipeline needs list of (name, transformer) tuples [OK]

Common Mistakes:

• Passing arguments without list brackets
• Using dict instead of list of tuples
• Omitting step names in pipeline

3. Given the following pipeline code, what will be the output of pipeline.transform([[0, 0], [1, 1]])?

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA

pipeline = Pipeline([
  ('scaler', StandardScaler()),
  ('pca', PCA(n_components=1))
])
pipeline.fit([[0, 0], [1, 1]])
result = pipeline.transform([[0, 0], [1, 1]])
print(result)

medium

A. Error: PCA requires more than one sample

B. [[0. 0.]
[1. 1.]]

C. [[0.5]
[0.5]]

D. [[-1.41421356]
[ 1.41421356]]

Solution

1. Step 1: Understand pipeline steps

   First, data is scaled to zero mean and unit variance, then PCA reduces to 1 component.

2. Step 2: Calculate transformed output

   Scaling [[0,0],[1,1]] centers data, PCA finds principal component; output is approximately [[-1.41421356],[1.41421356]].

3. Final Answer:

   [[-1.41421356] [ 1.41421356]] -> Option D

4. Quick Check:

   Scaling + PCA output = [[-1.41421356] [ 1.41421356]] [OK]

Hint: Scaling centers data; PCA output is principal component values [OK]

Common Mistakes:

• Expecting original data as output
• Confusing PCA output shape
• Assuming error due to small data

4. You have this pipeline code but it raises an error: ValueError: Expected 2D array, got 1D array instead. What is the likely cause?

pipeline = Pipeline([
  ('scaler', StandardScaler()),
  ('pca', PCA(n_components=1))
])
pipeline.fit([1, 2, 3, 4])

medium

A. Pipeline steps must be functions, not classes

B. Input to fit should be 2D array, not 1D list

C. StandardScaler requires integer inputs only

D. PCA cannot have n_components=1

Solution

1. Step 1: Analyze error message

   The error says input is 1D but 2D is expected for fit method.

2. Step 2: Check input format

   Input [1, 2, 3, 4] is a 1D list; fit expects 2D array like [[1], [2], [3], [4]].

3. Final Answer:

   Input to fit should be 2D array, not 1D list -> Option B

4. Quick Check:

   fit input shape must be 2D [OK]

Hint: fit() needs 2D array shape, not flat list [OK]

Common Mistakes:

• Passing 1D list instead of 2D array
• Misunderstanding PCA parameter limits
• Thinking StandardScaler restricts input types

5. You want to create a feature engineering pipeline that handles missing values by filling them with the median, then scales features, and finally selects the top 3 features using a model-based selector. Which pipeline setup is correct?

hard

A. Pipeline([('scaler', StandardScaler()), ('imputer', SimpleImputer(strategy='median')), ('selector', SelectFromModel(estimator=RandomForestClassifier(), max_features=3))])

B. Pipeline([('selector', SelectFromModel(estimator=RandomForestClassifier(), max_features=3)), ('imputer', SimpleImputer(strategy='median')), ('scaler', StandardScaler())])

C. Pipeline([('imputer', SimpleImputer(strategy='median')), ('scaler', StandardScaler()), ('selector', SelectFromModel(estimator=RandomForestClassifier(), max_features=3))])

D. Pipeline([('imputer', SimpleImputer(strategy='mean')), ('selector', SelectFromModel(estimator=RandomForestClassifier(), max_features=3)), ('scaler', StandardScaler())])

Solution

1. Step 1: Order pipeline steps logically

   Missing values must be handled first, then scaling, then feature selection.

2. Step 2: Check each option's correctness

   Pipeline([('imputer', SimpleImputer(strategy='median')), ('scaler', StandardScaler()), ('selector', SelectFromModel(estimator=RandomForestClassifier(), max_features=3))]) follows correct order and uses median for imputation. Pipeline([('scaler', StandardScaler()), ('imputer', SimpleImputer(strategy='median')), ('selector', SelectFromModel(estimator=RandomForestClassifier(), max_features=3))]) swaps imputer and scaler incorrectly. Pipeline([('selector', SelectFromModel(estimator=RandomForestClassifier(), max_features=3)), ('imputer', SimpleImputer(strategy='median')), ('scaler', StandardScaler())]) starts with selector which needs complete data. Pipeline([('imputer', SimpleImputer(strategy='mean')), ('selector', SelectFromModel(estimator=RandomForestClassifier(), max_features=3)), ('scaler', StandardScaler())]) uses mean instead of median and wrong order.

3. Final Answer:

   Pipeline([('imputer', SimpleImputer(strategy='median')), ('scaler', StandardScaler()), ('selector', SelectFromModel(estimator=RandomForestClassifier(), max_features=3))]) -> Option C

4. Quick Check:

   Impute -> scale -> select features [OK]

Hint: Impute missing -> scale -> select features in pipeline order [OK]

Common Mistakes:

• Placing scaler before imputer
• Selecting features before imputing missing values
• Using mean instead of median when median is required