Trend lines on scatter plots in Matplotlib - Step-by-Step Execution

Concept Flow - Trend lines on scatter plots

Start with data points

↓

Plot scatter points

↓

Calculate trend line coefficients

↓

Create line points using coefficients

↓

Plot trend line on scatter plot

↓

Show plot

We start with data points, plot them as a scatter plot, calculate the trend line, then draw it over the scatter plot.

Execution Sample

Matplotlib

import matplotlib.pyplot as plt
import numpy as np
x = np.array([1,2,3,4,5])
y = np.array([2,3,5,7,11])
plt.scatter(x, y)
coef = np.polyfit(x, y, 1)
plt.plot(x, coef[0]*x + coef[1])
plt.show()

This code plots points and draws a straight trend line fitting those points.

Execution Table

Step	Action	Variable/Expression	Result/Value
1	Create x array	x	[1 2 3 4 5]
2	Create y array	y	[2 3 5 7 11]
3	Plot scatter points	plt.scatter(x, y)	Scatter plot with 5 points
4	Calculate trend line coefficients	np.polyfit(x, y, 1)	coef = [2.2, -1.0]
5	Calculate trend line y values	coef[0]*x + coef[1]	[1.2 3.4 5.6 7.8 10. ]
6	Plot trend line	plt.plot(x, coef[0]*x + coef[1])	Line drawn over scatter
7	Show plot	plt.show()	Scatter plot with trend line displayed

💡 Plot displayed and execution ends

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 4	After Step 5	Final
x	undefined	[1 2 3 4 5]	[1 2 3 4 5]	[1 2 3 4 5]	[1 2 3 4 5]	[1 2 3 4 5]
y	undefined	undefined	[2 3 5 7 11]	[2 3 5 7 11]	[2 3 5 7 11]	[2 3 5 7 11]
coef	undefined	undefined	undefined	[2.2, -1.0]	[2.2, -1.0]	[2.2, -1.0]
trend_y	undefined	undefined	undefined	undefined	[1.2 3.4 5.6 7.8 10. ]	[1.2 3.4 5.6 7.8 10. ]

Key Moments - 3 Insights

Why does the trend line not pass through all scatter points exactly?

The trend line is a best fit line calculated by np.polyfit which minimizes overall distance, not necessarily passing through every point (see Step 4 and 5 in execution_table).

What do the two numbers in coef represent?

coef[0] is the slope (steepness) of the line, coef[1] is the intercept (where line crosses y-axis), shown in Step 4 of execution_table.

Why do we multiply coef[0] by x and add coef[1]?

This calculates y values on the trend line for each x, creating points to draw the line (Step 5 in execution_table).

Visual Quiz - 3 Questions

Look at the execution_table at Step 4, what does coef equal?

A [2.2, -1.0]

B [1, 0]

C [0, 1]

D [5, 11]

💡 Hint

Check the 'Result/Value' column at Step 4 in execution_table.

At which step do we calculate the y values for the trend line?

A Step 3

B Step 5

C Step 2

D Step 6

💡 Hint

Look for when coef[0]*x + coef[1] is computed in execution_table.

If we change the degree in np.polyfit from 1 to 2, what changes in the execution_table?

A Scatter points change

B x array changes

C coef will have 3 values instead of 2

D Plot will not show

💡 Hint

Think about polynomial degree and number of coefficients in Step 4.

Concept Snapshot

Trend lines on scatter plots:
- Use plt.scatter(x, y) to plot points
- Use np.polyfit(x, y, 1) to get line coefficients
- Calculate line y: slope*x + intercept
- Use plt.plot(x, line_y) to draw trend line
- Shows overall data trend visually

Full Transcript

We start with data points x and y. We plot these points as a scatter plot using plt.scatter. Then, we calculate the best fit line coefficients using np.polyfit with degree 1, which returns slope and intercept. Next, we compute y values on the trend line by multiplying slope with x and adding intercept. We plot this line over the scatter plot using plt.plot. Finally, we show the plot with plt.show. The trend line shows the general direction of the data but does not pass through all points exactly.

Practice

1. What is the main purpose of adding a trend line to a scatter plot in matplotlib?

easy

A. To show the overall pattern or relationship between variables

B. To add colors to the scatter points

C. To change the size of scatter points

D. To remove outliers from the data

Solution

1. Step 1: Understand scatter plot purpose

   A scatter plot shows individual data points but may not clearly show the overall trend.

2. Step 2: Role of trend line

   A trend line summarizes the general direction or pattern in the data, making relationships easier to see.

3. Final Answer:

   To show the overall pattern or relationship between variables -> Option A

4. Quick Check:

   Trend line = overall pattern [OK]

Hint: Trend lines summarize data direction, not point details [OK]

Common Mistakes:

• Thinking trend lines change point colors
• Confusing trend lines with point size changes
• Believing trend lines remove data points

2. Which of the following code snippets correctly fits a linear trend line to data arrays x and y using numpy?

easy

A. coefficients = np.polyfit(x, y, 1)

B. coefficients = np.polyfit(y, x, 1)

C. coefficients = np.polyfit(x, y, 2)

D. coefficients = np.polyfit(x, y)

Solution

1. Step 1: Understand np.polyfit parameters

   np.polyfit(x, y, degree) fits a polynomial of given degree to x and y data.

2. Step 2: Correct usage for linear fit

   For a linear trend line, degree = 1, and x is independent variable, y is dependent variable.

3. Final Answer:

   coefficients = np.polyfit(x, y, 1) -> Option A

4. Quick Check:

   np.polyfit(x, y, 1) fits linear trend [OK]

Hint: Use degree=1 for linear trend with np.polyfit(x, y, 1) [OK]

Common Mistakes:

• Swapping x and y in np.polyfit
• Omitting degree parameter
• Using degree 2 for linear trend

3. Given the code below, what will be the output of print(trendline(3))?

import numpy as np
x = np.array([1, 2, 3, 4])
y = np.array([2, 4, 6, 8])
coefficients = np.polyfit(x, y, 1)
trendline = np.poly1d(coefficients)
print(trendline(3))

medium

A. Error due to incorrect polyfit usage

B. 5.0

C. 9.0

D. 6.0

Solution

1. Step 1: Analyze data and polyfit

   Data y doubles x (2,4,6,8), so linear fit slope ~2 and intercept ~0.

2. Step 2: Calculate trendline at x=3

   trendline(3) = 2*3 + 0 = 6.0

3. Final Answer:

   6.0 -> Option D

4. Quick Check:

   trendline(3) = 6.0 [OK]

Hint: Check if y is roughly 2*x; then trendline(3) = 6 [OK]

Common Mistakes:

• Confusing x and y order in polyfit
• Forgetting poly1d creates function
• Assuming error instead of numeric output

4. Identify the error in the following code that tries to plot a scatter plot with a trend line:

import numpy as np
import matplotlib.pyplot as plt
x = np.array([1, 2, 3, 4])
y = np.array([2, 3, 5, 7])
coeff = np.polyfit(x, y, 1)
trend = np.poly1d(coeff)
plt.scatter(x, y)
plt.plot(x, trend)
plt.show()

medium

A. np.polyfit arguments are in wrong order

B. plt.plot(x, trend) should be plt.plot(x, trend(x))

C. plt.scatter should be called after plt.plot

D. Missing plt.xlabel and plt.ylabel calls

Solution

1. Step 1: Check trend line plotting

   plt.plot expects y-values, but trend is a function, not values.

2. Step 2: Correct function call

   Call trend(x) to get y-values for plotting: plt.plot(x, trend(x))

3. Final Answer:

   plt.plot(x, trend) should be plt.plot(x, trend(x)) -> Option B

4. Quick Check:

   Plot function needs y-values, not function [OK]

Hint: Call trend(x) to get y-values for plt.plot [OK]

Common Mistakes:

• Passing function instead of values to plt.plot
• Swapping x and y in polyfit
• Thinking plot order affects output
• Missing axis labels is not an error

5. You have noisy data with outliers. Which approach best improves the trend line fit on a scatter plot using matplotlib and numpy?

hard

A. Plot scatter without trend line to avoid confusion

B. Use np.polyfit with degree 1 on all data points

C. Remove outliers before fitting trend line with np.polyfit

D. Use plt.plot instead of plt.scatter for all points

Solution

1. Step 1: Understand effect of outliers

   Outliers can distort the trend line fit, making it less representative.

2. Step 2: Improve fit by removing outliers

   Removing or filtering outliers before fitting helps get a better trend line.

3. Final Answer:

   Remove outliers before fitting trend line with np.polyfit -> Option C

4. Quick Check:

   Remove outliers for better trend line [OK]

Hint: Filter outliers before fitting trend line for accuracy [OK]

Common Mistakes:

• Fitting trend line on all data including outliers
• Skipping trend line when data is noisy
• Confusing plot types for trend line fitting