KDE overlay concept in Matplotlib - Step-by-Step Execution

Concept Flow - KDE overlay concept

Start with raw data points

↓

Calculate KDE for dataset 1

↓

Calculate KDE for dataset 2

↓

Plot KDE curves on same graph

↓

Visualize overlapping density

↓

Interpret density peaks and overlaps

We start with raw data, compute KDE for each dataset, then plot them together to see how their densities overlap.

Execution Sample

Matplotlib

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import gaussian_kde

x1 = np.random.normal(0, 1, 100)
x2 = np.random.normal(2, 1, 100)

kde1 = gaussian_kde(x1)
kde2 = gaussian_kde(x2)

x = np.linspace(-4, 6, 200)
plt.plot(x, kde1(x), label='Dataset 1')
plt.plot(x, kde2(x), label='Dataset 2')
plt.legend()
plt.show()

This code creates two datasets, computes their KDEs, and plots both KDE curves on the same graph to show their density overlap.

Execution Table

Step	Action	Data/Variable	Result/Output
1	Generate dataset 1	x1	100 points from N(0,1)
2	Generate dataset 2	x2	100 points from N(2,1)
3	Compute KDE for dataset 1	kde1 = gaussian_kde(x1)	kde1 is a KDE function
4	Compute KDE for dataset 2	kde2 = gaussian_kde(x2)	kde2 is a KDE function
5	Create x values for plotting	x = np.linspace(-4,6,200)	Array of 200 points from -4 to 6
6	Evaluate kde1 on x	kde1(x)	Density values for dataset 1
7	Evaluate kde2 on x	kde2(x)	Density values for dataset 2
8	Plot kde1 and kde2	plt.plot	Two KDE curves overlayed
9	Show plot	plt.show()	Visual graph with KDE overlays
10	End	-	Execution complete

💡 All steps completed; KDE overlay plot displayed.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6	After Step 7	Final
x1	None	100 points N(0,1)	100 points N(0,1)	100 points N(0,1)	100 points N(0,1)	100 points N(0,1)	100 points N(0,1)	100 points N(0,1)	100 points N(0,1)
x2	None	None	100 points N(2,1)	100 points N(2,1)	100 points N(2,1)	100 points N(2,1)	100 points N(2,1)	100 points N(2,1)	100 points N(2,1)
kde1	None	None	None	KDE function for x1	KDE function for x1	KDE function for x1	KDE function for x1	KDE function for x1	KDE function for x1
kde2	None	None	None	None	KDE function for x2	KDE function for x2	KDE function for x2	KDE function for x2	KDE function for x2
x	None	None	None	None	None	Array from -4 to 6 (200 pts)	Array from -4 to 6 (200 pts)	Array from -4 to 6 (200 pts)	Array from -4 to 6 (200 pts)
kde1(x)	None	None	None	None	None	None	Density values (dataset 1)	Density values (dataset 1)	Density values (dataset 1)
kde2(x)	None	None	None	None	None	None	None	Density values (dataset 2)	Density values (dataset 2)

Key Moments - 3 Insights

Why do we use the same x values to evaluate both KDEs?

Using the same x values (step 5) ensures both KDE curves align on the same horizontal axis, making the overlay meaningful and comparable (see steps 6 and 7).

What does the KDE function represent after computation?

After steps 3 and 4, kde1 and kde2 are functions that estimate the probability density of the data, not just numbers. They can be evaluated at any x to get density values.

Why do the KDE curves overlap instead of being separate?

Because both KDEs are plotted on the same graph (step 8), their density curves can overlap where data distributions are close, showing areas of shared density.

Visual Quiz - 3 Questions

Look at the execution table at step 5. What does the variable x represent?

A The KDE function for dataset 1

B The original dataset 1 points

C An array of points from -4 to 6 used for plotting KDEs

D The density values of dataset 2

💡 Hint

Refer to step 5 in execution_table where x is created as linspace(-4,6,200).

At which step do we get the actual density values for dataset 2?

A Step 6

B Step 7

C Step 3

D Step 2

💡 Hint

Check step 7 in execution_table where kde2(x) is evaluated.

If we changed the range of x to linspace(-10, 10, 200), how would the execution_table change?

A Step 5's x values would range from -10 to 10 instead of -4 to 6

B Dataset 1 would have more points

C KDE functions would change to different distributions

D The plot would show only one KDE curve

💡 Hint

Look at step 5 where x is defined; changing linspace changes x values only.

Concept Snapshot

KDE overlay concept:
- Start with two datasets
- Compute KDE for each using gaussian_kde
- Use same x range to evaluate densities
- Plot both KDE curves on one graph
- Overlapping curves show density similarities
- Useful to compare distributions visually

Full Transcript

This visual execution traces the KDE overlay concept. We begin by generating two datasets from normal distributions. Then, we compute KDE functions for each dataset. Next, we create a common x array to evaluate both KDEs, ensuring they align on the same axis. We evaluate the KDE functions on this x array to get density values. Finally, we plot both KDE curves on the same graph to visualize where their densities overlap. This helps us compare the shape and spread of the two datasets visually. Key moments include understanding why the same x values are used for both KDEs, what the KDE functions represent, and why the curves overlap on the plot. The quiz checks understanding of variable roles and the effect of changing the x range.

Practice

1. What does a KDE overlay plot primarily help you understand when comparing multiple groups?

easy

A. How the data distributions overlap and differ smoothly

B. The exact count of data points in each group

C. The median value of each group only

D. The correlation between two variables

Solution

1. Step 1: Understand KDE overlay purpose

   KDE overlay plots show smooth curves representing data distributions for groups.

2. Step 2: Compare what KDE overlay reveals

   It highlights how distributions overlap or differ, not exact counts or medians.

3. Final Answer:

   How the data distributions overlap and differ smoothly -> Option A

4. Quick Check:

   KDE overlay = smooth distribution comparison [OK]

Hint: KDE overlay shows smooth curves, not counts or single values [OK]

Common Mistakes:

• Confusing KDE overlay with bar charts showing counts
• Thinking KDE overlay shows only median or mean
• Assuming KDE overlay shows correlation

2. Which of the following is the correct way to plot KDE overlays for two groups using seaborn?

easy

A. sns.kdeplot(data=df, x='value', group='group')

B. sns.kdeplot(data=df, y='value', color='group')

C. sns.kdeplot(data=df, x='value', hue='group', fill=True)

D. sns.kdeplot(data=df, x='value', label='group')

Solution

1. Step 1: Identify seaborn syntax for KDE overlays

   Use hue to separate groups and fill=True to shade areas.

2. Step 2: Check each option

   sns.kdeplot(data=df, x='value', hue='group', fill=True) uses correct parameters: hue='group' and fill=True. Others misuse parameters or omit grouping.

3. Final Answer:

   sns.kdeplot(data=df, x='value', hue='group', fill=True) -> Option C

4. Quick Check:

   Use hue for groups and fill for shading [OK]

Hint: Use hue=group and fill=True for KDE overlays in seaborn [OK]

Common Mistakes:

• Using 'group' instead of 'hue' parameter
• Confusing x and y parameters
• Omitting fill=True for shaded KDE

3. Given the code below, what will the plot show?

import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd

data = pd.DataFrame({
  'score': [1, 2, 2, 3, 4, 5, 5, 6, 7, 8],
  'group': ['A', 'A', 'B', 'B', 'A', 'B', 'A', 'B', 'A', 'B']
})
sns.kdeplot(data=data, x='score', hue='group', fill=True)
plt.show()

medium

A. Two smooth curves showing score distributions for groups A and B with shaded areas

B. A bar chart showing counts of scores for groups A and B

C. A scatter plot of scores colored by group

D. A line plot connecting scores in order for each group

Solution

1. Step 1: Understand sns.kdeplot with hue and fill

   The code plots kernel density estimates for 'score' by 'group' with shaded areas.

2. Step 2: Identify plot type from code

   It produces smooth curves (KDE) for groups A and B, not bars, scatter, or line plots.

3. Final Answer:

   Two smooth curves showing score distributions for groups A and B with shaded areas -> Option A

4. Quick Check:

   kdeplot with hue and fill = smooth shaded curves [OK]

Hint: kdeplot with hue and fill shows smooth shaded group distributions [OK]

Common Mistakes:

• Confusing KDE plot with bar or scatter plots
• Expecting raw data points instead of smooth curves
• Ignoring the fill parameter effect

4. The following code throws an error. What is the most likely cause?

import seaborn as sns
sns.kdeplot(data=df, x='value', hue='group', fill='yes')

medium

A. The data parameter cannot be a DataFrame

B. The fill parameter should be a boolean, not a string

C. The x parameter should be a list of columns

D. The hue parameter must be a list, not a string

Solution

1. Step 1: Check fill parameter type

   In seaborn's kdeplot, fill expects True or False, not a string like 'yes'.

2. Step 2: Validate other parameters

   Hue accepts a column name string, x accepts a column name string, data accepts DataFrame. So these are correct.

3. Final Answer:

   The fill parameter should be a boolean, not a string -> Option B

4. Quick Check:

   fill=True/False, not 'yes' [OK]

Hint: fill must be True or False, never a string [OK]

Common Mistakes:

• Passing 'yes' or 'no' as fill instead of True/False
• Misunderstanding hue parameter type
• Thinking data can't be a DataFrame

5. You want to compare the distribution of exam scores for three classes (X, Y, Z) using KDE overlays. Which approach best ensures clear visual comparison?

hard

A. Plot a single KDE curve for all classes combined without grouping

B. Plot histograms for each class on separate subplots without KDE

C. Plot KDE overlays but use the same color for all classes without labels

D. Plot KDE overlays with distinct colors, use hue='class', and add fill=True with transparency

Solution

1. Step 1: Identify best way to compare multiple groups with KDE

   Using hue='class' separates groups, distinct colors differentiate them, and fill=True with transparency helps see overlaps.

2. Step 2: Evaluate other options

   Separate histograms lose overlay comparison, single KDE hides group differences, same color without labels causes confusion.

3. Final Answer:

   Plot KDE overlays with distinct colors, use hue='class', and add fill=True with transparency -> Option D

4. Quick Check:

   Distinct colors + hue + fill = clear KDE overlay [OK]

Hint: Use hue, distinct colors, and fill with transparency for clarity [OK]

Common Mistakes:

• Plotting combined KDE without group separation
• Using same color for all groups causing confusion
• Not using fill or transparency to show overlaps