Feature union in ML Python

Introduction

Feature union helps combine different sets of features into one. It makes it easy to use many types of information together for better predictions.

When you have different types of data like text and numbers and want to use both.

When you want to try different ways to extract features and combine them.

When you want to build a pipeline that uses multiple feature extraction steps.

When you want to improve model accuracy by using more information.

When you want to keep your code clean by combining features in one step.

Syntax

ML Python

from sklearn.pipeline import FeatureUnion

feature_union = FeatureUnion(transformer_list=[
    ('name1', transformer1),
    ('name2', transformer2),
    # ...
])

combined_features = feature_union.fit_transform(X)

transformer_list is a list of (name, transformer) pairs.

Each transformer should have fit and transform methods.

Examples

This example combines PCA for numeric data and TF-IDF for text data.

ML Python

from sklearn.pipeline import FeatureUnion
from sklearn.decomposition import PCA
from sklearn.feature_extraction.text import TfidfVectorizer

union = FeatureUnion([
    ('pca', PCA(n_components=2)),
    ('tfidf', TfidfVectorizer())
])

Combines a custom length feature with a count vectorizer for text.

ML Python

from sklearn.preprocessing import FunctionTransformer
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.pipeline import FeatureUnion

union = FeatureUnion([
    ('length', FunctionTransformer(lambda x: x.apply(len).values.reshape(-1,1))),
    ('count', CountVectorizer())
])

Sample Model

This program loads iris data, scales it, applies PCA, and combines both features side by side.

ML Python

from sklearn.pipeline import FeatureUnion
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.datasets import load_iris

# Load sample data
iris = load_iris()
X = iris.data

# Define two feature transformers
scaler = StandardScaler()
pca = PCA(n_components=2)

# Combine features using FeatureUnion
union = FeatureUnion([
    ('scaled', scaler),
    ('pca', pca)
])

# Fit and transform data
X_combined = union.fit_transform(X)

print('Original shape:', X.shape)
print('Combined features shape:', X_combined.shape)
print('First 3 rows of combined features:')
print(X_combined[:3])

Important Notes

FeatureUnion concatenates features horizontally (side by side).

All transformers must output arrays with the same number of rows as input.

Use FeatureUnion inside pipelines to build complex workflows.

Summary

FeatureUnion combines multiple feature sets into one.

It helps use different data types or extraction methods together.

It keeps code organized and improves model input.

Practice

1. What is the main purpose of using FeatureUnion in machine learning?

easy

A. To combine multiple feature extraction methods into a single feature set

B. To split data into training and testing sets

C. To reduce the number of features by selecting the best ones

D. To train multiple models and average their predictions

Solution

1. Step 1: Understand FeatureUnion's role

   FeatureUnion is used to combine different feature extraction methods so their outputs join into one feature set.

2. Step 2: Compare with other options

   Splitting data, feature selection, and model averaging are different tasks not done by FeatureUnion.

3. Final Answer:

   To combine multiple feature extraction methods into a single feature set -> Option A

4. Quick Check:

   FeatureUnion = Combine features [OK]

Hint: FeatureUnion joins features, not data splits or models [OK]

Common Mistakes:

• Confusing FeatureUnion with data splitting
• Thinking it selects features instead of combining
• Mixing it up with model ensemble methods

2. Which of the following is the correct way to create a FeatureUnion with two transformers named 'tf1' and 'tf2'?

easy

A. FeatureUnion(tf1=transformer1, tf2=transformer2)

B. FeatureUnion({'tf1': transformer1, 'tf2': transformer2})

C. FeatureUnion([('tf1', transformer1), ('tf2', transformer2)])

D. FeatureUnion(transformer1, transformer2)

Solution

1. Step 1: Recall FeatureUnion syntax

   FeatureUnion expects a list of tuples, each tuple with a name and a transformer.

2. Step 2: Check each option

   FeatureUnion([('tf1', transformer1), ('tf2', transformer2)]) uses a list of tuples correctly. Options B, C, and D use wrong data structures or missing list.

3. Final Answer:

   FeatureUnion([('tf1', transformer1), ('tf2', transformer2)]) -> Option C

4. Quick Check:

   FeatureUnion needs list of (name, transformer) tuples [OK]

Hint: Use list of (name, transformer) tuples for FeatureUnion [OK]

Common Mistakes:

• Passing a dictionary instead of list of tuples
• Passing transformers without names
• Passing transformers as separate arguments

3. Given the code below, what will be the shape of X_transformed?

from sklearn.pipeline import FeatureUnion
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
import numpy as np

X = np.array([[1, 2, 3], [4, 5, 6]])

union = FeatureUnion([
    ('scale', StandardScaler()),
    ('pca', PCA(n_components=1))
])

X_transformed = union.fit_transform(X)

medium

A. (2, 1)

B. (2, 3)

C. (2, 2)

D. (2, 4)

Solution

1. Step 1: Analyze each transformer output

   StandardScaler keeps original shape (2 samples, 3 features) so output shape is (2,3). PCA with n_components=1 outputs (2,1).

2. Step 2: Combine outputs with FeatureUnion

   FeatureUnion concatenates outputs horizontally: (2,3) + (2,1) = (2,4).

3. Final Answer:

   (2, 4) -> Option D

4. Quick Check:

   Concatenate (2,3) and (2,1) = (2,4) [OK]

Hint: FeatureUnion concatenates horizontally, sum feature counts [OK]

Common Mistakes:

• Assuming PCA output replaces original features
• Thinking FeatureUnion stacks vertically
• Ignoring output shapes of individual transformers

4. You wrote this code but get an error:

union = FeatureUnion([
    ('scale', StandardScaler()),
    ('pca', PCA(n_components=3))
])

X_transformed = union.fit_transform([[1, 2], [3, 4], [5, 6]])

What is the likely cause of the error?

medium

A. PCA cannot have n_components greater than input features

B. StandardScaler requires 3D input, but input is 2D

C. FeatureUnion requires transformers to have fit_predict method

D. Input data must be a pandas DataFrame, not a list

Solution

1. Step 1: Check input data shape

   The input X = [[1,2],[3,4],[5,6]] has shape (3, 2), meaning 2 features.

2. Step 2: Analyze PCA configuration

   PCA(n_components=3) requests 3 components, but only 2 features are available, causing a ValueError.

3. Final Answer:

   PCA cannot have n_components greater than input features -> Option A

4. Quick Check:

   PCA n_components ≤ features [OK]

Hint: Check PCA n_components ≤ number of features [OK]

Common Mistakes:

• Assuming StandardScaler needs 3D input
• Thinking FeatureUnion needs fit_predict
• Believing input must be DataFrame

5. You want to combine text and numeric features for a model. You have a TfidfVectorizer for text and StandardScaler for numeric data. How do you use FeatureUnion to prepare the data correctly?

hard

A. Apply TfidfVectorizer and StandardScaler separately, then add their outputs manually

B. Use FeatureUnion with transformers for text and numeric, each applied to their columns via ColumnTransformer

C. Use FeatureUnion directly on raw data without preprocessing

D. Use StandardScaler on text data and TfidfVectorizer on numeric data

Solution

1. Step 1: Understand data types and transformers

   Text and numeric data need different preprocessing. TfidfVectorizer works on text, StandardScaler on numeric features.

2. Step 2: Use ColumnTransformer with FeatureUnion

   Apply each transformer to correct columns using ColumnTransformer, then combine with FeatureUnion to merge features.

3. Final Answer:

   Use FeatureUnion with transformers for text and numeric, each applied to their columns via ColumnTransformer -> Option B

4. Quick Check:

   Separate preprocessing per data type, then combine [OK]

Hint: Preprocess each data type separately, then combine features [OK]

Common Mistakes:

• Applying wrong transformer to wrong data type
• Skipping column selection before FeatureUnion
• Trying to combine raw data without preprocessing