Descriptive statistics (mean, median, std) in MATLAB - Deep Dive

Overview - Descriptive statistics (mean, median, std)

What is it?

Descriptive statistics are simple numbers that summarize a set of data. They help us understand the center, spread, and shape of the data. The mean is the average value, the median is the middle value when data is sorted, and the standard deviation shows how spread out the data is. These measures give a quick snapshot of what the data looks like.

Why it matters

Without descriptive statistics, we would have to look at every single data point to understand a dataset, which is slow and confusing. These statistics help us quickly grasp important features like typical values and variability. This is crucial in many fields like science, business, and health to make decisions based on data patterns.

Where it fits

Before learning descriptive statistics, you should know basic data types and how to collect data. After this, you can learn about data visualization and inferential statistics, which use these summaries to make predictions or test ideas.

Mental Model

Core Idea

Descriptive statistics reduce complex data into simple numbers that describe its center and spread.

Think of it like...

Imagine you have a basket of apples of different sizes. The mean is like the average size of all apples, the median is the size of the apple in the middle if you lined them up from smallest to largest, and the standard deviation tells you how much the sizes vary from the average.

Data Set: [3, 7, 7, 19, 24]

Sorted: 3 ─ 7 ─ 7 ─ 19 ─ 24

Mean = (3+7+7+19+24)/5 = 12
Median = 7 (middle value)
Std Dev = measure of spread around 12

┌───────────────┐
│ 3 7 7 19 24   │
│   ↑   ↑       │
│ Median Mean   │
└───────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding the Mean

Concept: The mean is the average value of a dataset, found by adding all numbers and dividing by how many there are.

Suppose you have data points: 4, 8, 6, 10. To find the mean: 1. Add all numbers: 4 + 8 + 6 + 10 = 28 2. Count numbers: 4 3. Divide sum by count: 28 / 4 = 7 In MATLAB, use mean([4,8,6,10]) to get 7.

Result

The mean is 7, representing the average value of the data.

Understanding the mean helps you find a single value that represents the whole dataset's center.

2

Foundation

Finding the Median

Concept: The median is the middle value when data is sorted, showing the center without being affected by extreme values.

Given data: 3, 1, 9, 7, 5 1. Sort data: 1, 3, 5, 7, 9 2. Find middle value: 5 (third number) If even count, median is average of two middle numbers. In MATLAB, use median([3,1,9,7,5]) to get 5.

Result

The median is 5, the middle value that splits data into two halves.

Median is useful when data has outliers because it shows the center without being pulled by very large or small values.

3

Intermediate

Calculating Standard Deviation

🤔Before reading on: do you think standard deviation measures average distance from the mean or the range of data? Commit to your answer.

Concept: Standard deviation measures how spread out data points are from the mean, showing variability.

Data: 2, 4, 4, 4, 5, 5, 7, 9 Steps: 1. Find mean: (2+4+4+4+5+5+7+9)/8 = 5 2. Subtract mean and square: (2-5)^2=9, (4-5)^2=1, ... 3. Average squared differences: sum/8 = 4 4. Take square root: sqrt(4) = 2 In MATLAB, std([2,4,4,4,5,5,7,9]) returns 2.

Result

Standard deviation is 2, indicating typical distance from the mean.

Knowing standard deviation helps you understand if data points are close to the average or widely spread.

4

Intermediate

Using MATLAB Functions for Statistics

🤔Before reading on: do you think MATLAB functions like mean, median, and std work only on vectors or also on matrices? Commit to your answer.

Concept: MATLAB provides built-in functions to calculate mean, median, and standard deviation easily on arrays and matrices.

Example vector: data = [1, 3, 5, 7, 9]; mean(data) returns 5 median(data) returns 5 std(data) returns about 3.1623 For matrices, these functions operate column-wise by default. Example: M = [1 2 3; 4 5 6]; mean(M) returns [2.5 3.5 4.5] Use mean(M, 'all') for overall mean.

Result

MATLAB functions quickly compute statistics for vectors and matrices.

Using built-in functions saves time and reduces errors compared to manual calculations.

5

Intermediate

Comparing Mean and Median

🤔Before reading on: in a dataset with extreme values, do you think mean or median better represents the center? Commit to your answer.

Concept: Mean is sensitive to extreme values (outliers), while median is more robust and shows the middle value.

Data with outlier: [1, 2, 3, 4, 100] Mean = (1+2+3+4+100)/5 = 22 Median = 3 The mean is pulled up by 100, but median stays central. This shows median better represents typical value when outliers exist.

Result

Median is 3, mean is 22; median better reflects center here.

Knowing when to use median instead of mean prevents misleading conclusions from skewed data.

6

Advanced

Standard Deviation vs Variance

🤔Before reading on: do you think variance or standard deviation is easier to interpret in the original data units? Commit to your answer.

Concept: Variance is the average squared deviation from the mean; standard deviation is its square root, returning to original units.

Variance calculation: Data: 3, 7, 7, 19, 24 Mean = 12 Squared differences: (3-12)^2=81, (7-12)^2=25, ... Variance = average of squared differences Standard deviation = sqrt(variance) Standard deviation is easier to understand because it uses the same units as data.

Result

Standard deviation gives spread in original data units, variance in squared units.

Understanding the difference helps interpret variability correctly and communicate results clearly.

7

Expert

Handling NaN Values in Statistics

🤔Before reading on: do you think MATLAB's mean, median, and std ignore NaN values by default or return NaN? Commit to your answer.

Concept: By default, MATLAB functions return NaN if data contains NaN values; special options or functions are needed to ignore them.

Example: data = [1, 2, NaN, 4]; mean(data) returns NaN Use mean(data, 'omitnan') to ignore NaNs and get 2.3333 Similarly, median(data, 'omitnan') and std(data, 'omitnan') work. This is important when datasets have missing values.

Result

Using 'omitnan' option returns statistics ignoring missing data.

Knowing how to handle NaNs prevents incorrect results and errors in real-world messy data.

Under the Hood

MATLAB calculates mean by summing all elements and dividing by count. Median is found by sorting data and selecting the middle element(s). Standard deviation is computed by first finding the mean, then calculating squared differences from the mean, averaging them (variance), and finally taking the square root. Internally, MATLAB uses optimized vectorized operations for speed and handles matrices column-wise by default.

Why designed this way?

These statistics were designed to summarize data simply and effectively. Mean is intuitive but sensitive to outliers, so median was introduced to provide a robust center measure. Standard deviation quantifies spread in original units, making interpretation easier than variance. MATLAB's design prioritizes ease of use and performance, offering built-in functions that handle common cases and missing data flexibly.

┌───────────────┐
│   Data Array  │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│   mean()      │
│ sum(data)/n   │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│   median()    │
│ sort & pick   │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│   std()       │
│ sqrt(avg sq.  │
│ diff from mean)│
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does the mean always represent the most common value in data? Commit to yes or no.

Common Belief: The mean is the most frequent or typical value in the dataset.

Reality: The mean is the average, which can be influenced by extreme values and may not be the most common value (mode).

Why it matters: Relying on mean as the typical value can mislead decisions, especially with skewed data.

Quick: Do you think median is always equal to mean in symmetric data? Commit to yes or no.

Common Belief: Median and mean are always the same if data is symmetric.

Reality: Median and mean are equal only in perfectly symmetric distributions; small differences can exist in real data.

Why it matters: Assuming equality can cause errors in interpreting data center and skewness.

Quick: Does a higher standard deviation always mean data has more outliers? Commit to yes or no.

Common Belief: A large standard deviation means there are many outliers in the data.

Reality: Standard deviation measures spread, but high spread can come from natural variability, not just outliers.

Why it matters: Misinterpreting spread as outliers can lead to wrong data cleaning or analysis steps.

Quick: Do MATLAB's mean, median, and std functions ignore NaN values by default? Commit to yes or no.

Common Belief: These functions automatically ignore NaN values without extra options.

Reality: By default, MATLAB returns NaN if any NaN is present; you must specify 'omitnan' to ignore them.

Why it matters: Not handling NaNs properly can cause unexpected NaN results and analysis failures.

Expert Zone

1

Mean is sensitive to data scale and units; rescaling data changes mean but not median.

2

Standard deviation assumes data is roughly symmetric; for skewed data, other spread measures like interquartile range may be better.

3

MATLAB's column-wise default for matrices can cause confusion; specifying dimension explicitly avoids bugs.

When NOT to use

Avoid using mean for heavily skewed data or data with many outliers; use median or trimmed means instead. For spread, consider robust measures like median absolute deviation when data is not normal. When data has missing values, always use 'omitnan' options or preprocess data to avoid NaN contamination.

Production Patterns

In real-world data pipelines, descriptive statistics are used for quick data quality checks, feature engineering, and anomaly detection. Automated scripts compute these stats on large datasets to monitor changes over time. Experts combine these with visualization and robust statistics to guide modeling decisions.

Connections

Data Visualization

Descriptive statistics summarize data that visualization then displays graphically.

Knowing descriptive statistics helps interpret charts like histograms and boxplots by linking numbers to shapes.

Inferential Statistics

Descriptive statistics provide the foundation for inferential methods that make predictions or test hypotheses.

Understanding mean, median, and standard deviation is essential before learning confidence intervals or hypothesis tests.

Quality Control in Manufacturing

Descriptive statistics are used to monitor product measurements and detect variation in processes.

Recognizing how mean and standard deviation track consistency helps improve real-world production quality.

Common Pitfalls

#1 Using mean to summarize skewed data with outliers.

Wrong approach:

data = [1, 2, 3, 4, 100];
avg = mean(data); % avg = 22

Correct approach:

data = [1, 2, 3, 4, 100];
center = median(data); % center = 3

Root cause:

Misunderstanding that mean is sensitive to extreme values and may not represent typical data.

#2 Ignoring NaN values without specifying options in MATLAB functions.

Wrong approach:

data = [1, 2, NaN, 4];
avg = mean(data); % returns NaN

Correct approach:

data = [1, 2, NaN, 4];
avg = mean(data, 'omitnan'); % returns 2.3333

Root cause:

Assuming MATLAB functions automatically handle missing data.

#3 Applying std to a matrix without specifying dimension and misinterpreting results.

Wrong approach:

M = [1 2 3; 4 5 6];
spread = std(M); % computes std column-wise

Correct approach:

M = [1 2 3; 4 5 6];
spread = std(M, 0, 'all'); % computes std over all elements

Root cause:

Not knowing MATLAB defaults to column-wise operations for matrices.

Key Takeaways

Descriptive statistics like mean, median, and standard deviation summarize data simply and effectively.

Mean is the average but sensitive to outliers; median is the middle value and more robust to extremes.

Standard deviation measures how spread out data is around the mean, helping understand variability.

MATLAB provides built-in functions to calculate these statistics quickly, but handling NaN values requires care.

Choosing the right statistic depends on data shape and presence of outliers to avoid misleading conclusions.

Practice

1. What does the mean function calculate in MATLAB when applied to a numeric array?

easy

A. The difference between the largest and smallest values

B. The middle value after sorting the array

C. The total sum of all elements in the array

D. The average value of all elements in the array

Solution

1. Step 1: Understand the mean function

   The mean function calculates the average by adding all elements and dividing by the number of elements.

2. Step 2: Compare with other statistics

   Median is the middle value, not the average. Sum is total, not average. Range is difference, not average.

3. Final Answer:

   The average value of all elements in the array -> Option D

4. Quick Check:

   Mean = Average [OK]

Hint: Mean is the sum divided by count [OK]

Common Mistakes:

• Confusing mean with median
• Thinking mean is the sum only
• Mixing mean with range

2. Which of the following is the correct MATLAB syntax to calculate the standard deviation of a vector data?

easy

A. median(data)

B. standard_deviation(data)

C. std(data)

D. mean(data)

Solution

1. Step 1: Recall MATLAB function names

   MATLAB uses std() to calculate standard deviation, not standard_deviation().

2. Step 2: Eliminate incorrect options

   mean() calculates average, median() calculates middle value, so both are incorrect for standard deviation.

3. Final Answer:

   std(data) -> Option C

4. Quick Check:

   Standard deviation function is std() [OK]

Hint: Use std() for standard deviation in MATLAB [OK]

Common Mistakes:

• Using mean() instead of std()
• Typing non-existent function standard_deviation()
• Confusing median() with std()

3. What is the output of the following MATLAB code?

data = [3, 7, 5, 9, 1];
m = median(data);
s = std(data);
[m, s]

medium

A. [5, 2.8284]

B. [5, 3.1623]

C. [7, 3.1623]

D. [3, 2.8284]

Solution

1. Step 1: Calculate the median of data

   Sorted data is [1, 3, 5, 7, 9]. The middle value is 5, so median = 5.

2. Step 2: Calculate the standard deviation

   Mean = (3+7+5+9+1)/5 = 5. Differences squared: (3-5)^2=4, (7-5)^2=4, (5-5)^2=0, (9-5)^2=16, (1-5)^2=16. Sum = 40, variance = 40/4 = 10 (using n-1 for std), std = sqrt(10) ≈ 3.1623.

3. Final Answer:

   [5, 3.1623] -> Option B

4. Quick Check:

   Median = 5, Std ≈ 3.16 [OK]

Hint: Median is middle; std uses n-1 in denominator [OK]

Common Mistakes:

• Using n instead of n-1 for std denominator
• Mixing mean and median values
• Incorrectly sorting data for median

4. Identify the error in this MATLAB code snippet that calculates the mean and median of values:

values = [10, 20, 30, 40];
mean_val = mean(values)
median_val = median(values

medium

A. Missing closing parenthesis in median function call

B. Using mean instead of median for median_val

C. Incorrect variable name 'values'

D. No error, code runs fine

Solution

1. Step 1: Check syntax of median function call

   The line median_val = median(values is missing a closing parenthesis.

2. Step 2: Confirm other parts are correct

   mean(values) is correct, variable name is consistent, so no other errors.

3. Final Answer:

   Missing closing parenthesis in median function call -> Option A

4. Quick Check:

   Syntax error due to missing ')' [OK]

Hint: Check all parentheses in function calls [OK]

Common Mistakes:

• Ignoring missing parenthesis errors
• Confusing mean and median functions
• Assuming variable name errors without checking

5. Given a dataset data = [2, 4, 4, 4, 5, 5, 7, 9], which MATLAB code correctly calculates the mean, median, and standard deviation and stores them in variables m, med, s respectively?

hard

A. m = mean(data); med = median(data); s = std(data);

B. m = median(data); med = mean(data); s = std(data);

C. m = std(data); med = mean(data); s = median(data);

D. m = mean(data); med = std(data); s = median(data);

Solution

1. Step 1: Match variables to correct functions

   Variable m should hold mean, med median, s standard deviation.

2. Step 2: Verify correct function usage

   m = mean(data); med = median(data); s = std(data); assigns mean to m, median to med, std to s, which is correct.

3. Final Answer:

   m = mean(data); med = median(data); s = std(data); -> Option A

4. Quick Check:

   Correct variable-function mapping [OK]

Hint: Assign mean, median, std to matching variable names [OK]

Common Mistakes:

• Swapping mean and median assignments
• Assigning std to median variable
• Mixing variable names and functions