Gaussian Mixture Models in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Gaussian Mixture Model Components

What does each component in a Gaussian Mixture Model (GMM) represent?

A A neural network layer used for feature extraction

B A decision boundary separating classes in the dataset

C A single Gaussian distribution representing a cluster in the data

D A single data point used as a centroid

💡 Hint

Think about how GMM models data distribution using simpler parts.

Each component in a GMM is a Gaussian distribution that models one cluster or group in the data. The overall model is a weighted sum of these Gaussians.

❓ Predict Output

intermediate

Output of GMM Prediction Probabilities

What is the output of the following Python code using sklearn's GaussianMixture?

ML Python

from sklearn.mixture import GaussianMixture
import numpy as np

X = np.array([[0], [1], [2], [3]])
gmm = GaussianMixture(n_components=2, random_state=0)
gmm.fit(X)
probs = gmm.predict_proba([[1.5]])
print(probs)

A [[0.5 0.5]]

B [[0.3 0.7]]

C [[0.7 0.3]]

D Raises a ValueError due to input shape

💡 Hint

Predict_proba returns the probability of the sample belonging to each component.

The output shows the probabilities that the input point belongs to each Gaussian component. The exact values depend on the fitted model but here the point is closer to the second component, so higher probability is for component 1 (index 1).

❓ Model Choice

advanced

Choosing Number of Components in GMM

You want to cluster data with unknown groups using GMM. Which method helps select the best number of components?

A Always use 2 components for simplicity

B Pick the number of components equal to the number of data points

C Use the mean squared error between data points and cluster centers

D Use the Bayesian Information Criterion (BIC) to compare models with different components

💡 Hint

Think about a criterion that balances model fit and complexity.

BIC helps select the number of components by penalizing model complexity while rewarding better fit. Lower BIC means a better model choice.

❓ Hyperparameter

advanced

Effect of Covariance Type in GMM

What is the effect of setting the covariance_type parameter to 'diag' in a GaussianMixture model?

A Each component has its own diagonal covariance matrix, allowing different variances per feature but no covariance

B All components share the same full covariance matrix

C Covariance matrices are fixed to identity matrices

D Covariance matrices are scalar multiples of the identity matrix

💡 Hint

Diagonal covariance means no correlation between features within each component.

Setting covariance_type='diag' means each Gaussian component has its own covariance matrix but only the diagonal elements are non-zero, modeling variance per feature independently.

❓ Metrics

expert

Evaluating GMM Clustering Quality

Which metric is most appropriate to evaluate the quality of clusters found by a Gaussian Mixture Model when true labels are unknown?

A Silhouette score measuring how similar an object is to its own cluster compared to other clusters

B Accuracy score comparing predicted labels to true labels

C Mean squared error between data points and cluster centers

D Log loss computed from predicted probabilities and true labels

💡 Hint

Think about a metric that works without knowing true labels.

Silhouette score measures cluster cohesion and separation without needing true labels, making it suitable for unsupervised clustering evaluation.

Practice

1. What is the main idea behind a Gaussian Mixture Model (GMM)?

easy

A. It assumes data is made of several bell-shaped groups mixed together.

B. It uses decision trees to split data into groups.

C. It finds the single best line to fit the data points.

D. It clusters data by measuring distances only.

Solution

1. Step 1: Understand GMM concept

   GMM assumes data comes from multiple groups, each shaped like a bell curve (Gaussian).

2. Step 2: Compare with other methods

   Unlike decision trees or distance-only methods, GMM models overlapping groups with probabilities.

3. Final Answer:

   It assumes data is made of several bell-shaped groups mixed together. -> Option A

4. Quick Check:

   GMM = mixture of Gaussians [OK]

Hint: Remember GMM = mix of bell curves for groups [OK]

Common Mistakes:

• Confusing GMM with decision trees
• Thinking GMM finds one line only
• Assuming GMM uses only distances

2. Which Python library provides a built-in Gaussian Mixture Model class?

easy

A. matplotlib

B. pandas

C. scikit-learn

D. tensorflow

Solution

1. Step 1: Identify libraries for ML models

   scikit-learn is a popular library with many ML models including GMM.

2. Step 2: Check other libraries' purpose

   matplotlib is for plotting, pandas for data handling, tensorflow for deep learning, not GMM specifically.

3. Final Answer:

   scikit-learn -> Option C

4. Quick Check:

   GMM in scikit-learn [OK]

Hint: GMM class is in scikit-learn, not plotting or deep learning libs [OK]

Common Mistakes:

• Choosing matplotlib for modeling
• Confusing pandas with ML models
• Picking tensorflow for GMM

3. What will the following Python code output?

from sklearn.mixture import GaussianMixture
import numpy as np
X = np.array([[1], [2], [3], [10], [11], [12]])
gmm = GaussianMixture(n_components=2, random_state=0)
gmm.fit(X)
labels = gmm.predict(X)
print(labels.tolist())

medium

A. [1, 0, 1, 0, 1, 0]

B. [0, 0, 0, 1, 1, 1]

C. [0, 1, 0, 1, 0, 1]

D. [1, 1, 1, 0, 0, 0]

Solution

1. Step 1: Understand data and model

   Data has two clear groups: near 1-3 and near 10-12. GMM with 2 components fits these groups.

2. Step 2: Predict labels

   GMM assigns first three points to one group (label 0) and last three to another (label 1).

3. Final Answer:

   [0, 0, 0, 1, 1, 1] -> Option B

4. Quick Check:

   Groups split as low and high values [OK]

Hint: GMM labels cluster points close together [OK]

Common Mistakes:

• Mixing label order (0 vs 1)
• Assuming alternating labels
• Ignoring clear group separation

4. Identify the error in this GMM code snippet:

from sklearn.mixture import GaussianMixture
X = [[1, 2], [3, 4], [5, 6]]
gmm = GaussianMixture(n_components=2)
gmm.fit(X)
labels = gmm.predict(X)
print(labels)

medium

A. GaussianMixture requires a random_state parameter.

B. n_components must be 3 or more for this data.

C. fit() method should be called after predict().

D. X should be a NumPy array, not a list of lists.

Solution

1. Step 1: Check data format for GMM

   GMM expects input as a NumPy array, not a plain Python list.

2. Step 2: Verify other parameters and method order

   n_components=2 is valid, random_state is optional, fit() must be before predict().

3. Final Answer:

   X should be a NumPy array, not a list of lists. -> Option D

4. Quick Check:

   Input data type matters for GMM [OK]

Hint: Use NumPy arrays for GMM input data [OK]

Common Mistakes:

• Passing lists instead of arrays
• Wrong order of fit and predict
• Thinking random_state is mandatory

5. You have a dataset with overlapping groups of different sizes and shapes. Which advantage of Gaussian Mixture Models makes them suitable here?

hard

A. They can model overlapping groups with different shapes using probabilities.

B. They always create groups of equal size.

C. They only work for groups that are perfectly separated.

D. They require groups to be circular and same size.

Solution

1. Step 1: Understand group overlap and shape

   Real data groups often overlap and differ in shape and size.

2. Step 2: Match GMM strengths

   GMM uses probabilities to model overlapping groups with different shapes, unlike simpler methods.

3. Final Answer:

   They can model overlapping groups with different shapes using probabilities. -> Option A

4. Quick Check:

   GMM handles overlap and shape variation [OK]

Hint: GMM models overlap and shape differences well [OK]

Common Mistakes:

• Thinking GMM needs equal group sizes
• Assuming groups must be separate
• Believing GMM only fits circular groups