Probability density and cumulative functions in SciPy - Deep Dive

Overview - Probability density and cumulative functions

What is it?

Probability density and cumulative functions describe how likely different outcomes are in a continuous random process. The probability density function (PDF) shows the relative likelihood of a value occurring at each point. The cumulative distribution function (CDF) shows the total probability of a value being less than or equal to a point. Together, they help us understand and work with continuous data in statistics.

Why it matters

Without these functions, we couldn't measure or predict how continuous data behaves, like heights, temperatures, or test scores. They let us calculate probabilities for ranges of values, which is essential for decision-making, risk assessment, and scientific analysis. Without them, we would only guess or rely on incomplete information.

Where it fits

Before learning this, you should understand basic probability and random variables. After this, you can explore statistical inference, hypothesis testing, and machine learning models that use probability distributions.

Mental Model

Core Idea

The PDF shows how dense the probability is at each point, while the CDF sums these probabilities up to a point to show total chance so far.

Think of it like...

Imagine pouring sand on a table shaped like a curve. The PDF is how thick the sand is at each spot, and the CDF is how much sand has piled up from the start to any spot.

Probability Functions

PDF (Probability Density Function):
  Height of curve at each x shows likelihood density

CDF (Cumulative Distribution Function):
  Increases from 0 to 1 as x moves right

  1 ┤                 ╭───────
    │                ╭╯
 0.5┤          ╭─────╯
    │         ╭╯
  0 ┼─────────╯────────────
     0       x

PDF curve is the slope of the CDF curve at each point.

Build-Up - 7 Steps

1

Foundation

Understanding continuous random variables

Concept: Continuous random variables can take any value in a range, unlike discrete variables which have separate values.

A continuous random variable might be the height of people in a room. It can be 170.1 cm, 170.12 cm, or any value in between. We can't list all possible values because there are infinitely many.

Result

You understand that continuous variables need special tools to describe their probabilities because you can't count outcomes one by one.

Knowing the difference between continuous and discrete variables is key to choosing the right probability tools.

2

Foundation

What is a probability density function (PDF)?

Concept: The PDF shows how probability is spread over values for a continuous variable.

The PDF is a curve where the area under the curve between two points gives the chance the variable falls in that range. The curve itself is not a probability but a density. The total area under the curve is always 1.

Result

You can find the chance of a value falling in any range by calculating the area under the PDF curve for that range.

Understanding that probability is area under the curve, not the height itself, prevents confusion.

3

Intermediate

What is a cumulative distribution function (CDF)?

Concept: The CDF gives the total probability that the variable is less than or equal to a value.

The CDF at a point x is the area under the PDF curve from the smallest value up to x. It starts at 0 and rises to 1 as x moves to the maximum possible value.

Result

You can quickly find the chance that a value is below a threshold by looking at the CDF.

Knowing the CDF accumulates probability helps in understanding percentiles and thresholds.

4

Intermediate

Using scipy to compute PDF and CDF

🤔Before reading on: do you think scipy requires you to manually integrate the PDF to get the CDF? Commit to your answer.

Concept: scipy provides built-in functions to calculate PDF and CDF for many distributions without manual integration.

For example, to get PDF and CDF of a normal distribution at x=0, use: from scipy.stats import norm pdf_value = norm.pdf(0) cdf_value = norm.cdf(0) These functions return the density and cumulative probability directly.

Result

You get numeric values for PDF and CDF instantly, making analysis easier and less error-prone.

Knowing scipy handles integration internally saves time and avoids mistakes.

5

Intermediate

Visualizing PDF and CDF with scipy

🤔Before reading on: do you think the CDF curve looks like a smooth step or a smooth curve? Commit to your answer.

Concept: Plotting PDF and CDF helps understand their shapes and relationship.

Using matplotlib and scipy, you can plot PDF and CDF: import numpy as np import matplotlib.pyplot as plt from scipy.stats import norm x = np.linspace(-4, 4, 100) pdf = norm.pdf(x) cdf = norm.cdf(x) plt.plot(x, pdf, label='PDF') plt.plot(x, cdf, label='CDF') plt.legend() plt.show()

Result

You see the bell-shaped PDF curve and the smooth S-shaped CDF curve.

Visualizing both functions side-by-side clarifies how the CDF accumulates the PDF.

6

Advanced

Interpreting PDF and CDF for decision making

🤔Before reading on: do you think the PDF or CDF is better for finding the probability between two values? Commit to your answer.

Concept: The CDF can be used to find probabilities between two points by subtraction, while the PDF shows density at points.

To find the chance that a value lies between a and b: P(a < X ≤ b) = CDF(b) - CDF(a) This is easier than integrating the PDF manually. For example: prob = norm.cdf(1) - norm.cdf(-1) This gives the probability that X is between -1 and 1.

Result

You can calculate range probabilities quickly and accurately using CDF differences.

Understanding how to use CDF differences simplifies probability calculations for intervals.

7

Expert

Handling edge cases and numerical stability

🤔Before reading on: do you think PDF values can be greater than 1? Commit to your answer.

Concept: PDF values can be greater than 1 if the distribution is very narrow, but total area remains 1. Also, numerical methods in scipy handle edge cases carefully.

For example, a normal distribution with very small standard deviation has a tall PDF peak. PDF values can exceed 1, but the area under the curve is still 1. scipy uses algorithms to avoid floating-point errors when computing CDF and PDF at extreme values, ensuring stable results.

Result

You get accurate PDF and CDF values even for extreme inputs or narrow distributions.

Knowing PDF height can exceed 1 prevents confusion, and trusting scipy's numerical methods avoids manual errors.

Under the Hood

The PDF is the derivative of the CDF, meaning the CDF is the integral (area under the curve) of the PDF. Internally, scipy uses mathematical formulas or numerical integration to compute these functions efficiently. For standard distributions, closed-form formulas are used. For others, numerical methods approximate the values. The CDF accumulates probability from the left, ensuring it is always between 0 and 1 and non-decreasing.

Why designed this way?

Probability theory defines continuous distributions with PDFs and CDFs to handle infinite possible values. Using derivatives and integrals connects these functions mathematically. scipy implements these to provide fast, accurate, and reliable calculations for many distributions, avoiding manual integration and reducing errors.

Continuous Distribution Functions

  PDF (f(x))
  ┌─────────────┐
  │             │
  │   ╭───╮     │
  │  ╭╯   ╰╮    │
  │ ╭╯     ╰╮   │
  └─╯       ╰───┘

  CDF (F(x))
  ┌─────────────┐
  │             │
  │      ╭────╮ │
  │    ╭─╯    ╰─╮
  │  ╭─╯        ╰
  └─╯            

Relationship:
  F(x) = ∫ f(t) dt from -∞ to x
  f(x) = dF(x)/dx

Myth Busters - 3 Common Misconceptions

Quick: Do you think the PDF value at a point is the probability of that exact value? Commit to yes or no.

Common Belief: The PDF value at a point gives the probability that the variable equals that exact value.

Reality: For continuous variables, the probability at any exact point is zero. The PDF value is a density, not a probability.

Why it matters: Mistaking PDF values for probabilities leads to wrong conclusions and misuse of probability concepts.

Quick: Do you think the CDF can decrease at some points? Commit to yes or no.

Common Belief: The CDF can go down if the PDF is negative in some regions.

Reality: The CDF is always non-decreasing because probabilities accumulate and cannot decrease.

Why it matters: Expecting a decreasing CDF can cause confusion and errors in interpreting probabilities.

Quick: Do you think PDF values must always be less than or equal to 1? Commit to yes or no.

Common Belief: PDF values cannot be greater than 1 because probabilities are between 0 and 1.

Reality: PDF values can be greater than 1 if the distribution is very narrow; what matters is the total area under the curve equals 1.

Why it matters: Misunderstanding this can cause learners to wrongly reject valid PDFs or misinterpret density.

Expert Zone

1

PDF values represent density, not probability, so comparing PDF heights across different distributions requires care.

2

Numerical computation of CDFs for extreme values uses special algorithms to avoid floating-point underflow or overflow.

3

Some distributions have no closed-form PDF or CDF, requiring numerical approximation methods that trade off speed and accuracy.

When NOT to use

PDF and CDF are not suitable for discrete variables; use probability mass functions (PMF) instead. For very complex or unknown distributions, consider kernel density estimation or empirical distribution functions.

Production Patterns

In real-world systems, PDFs and CDFs are used for risk modeling, anomaly detection, and probabilistic forecasting. Professionals often precompute CDF lookup tables for speed or use vectorized scipy functions for batch processing large datasets.

Connections

Integral calculus

PDF and CDF are connected through integration and differentiation.

Understanding integrals and derivatives helps grasp how probability accumulates and how densities relate to totals.

Signal processing

PDFs resemble signal amplitude distributions; CDFs relate to cumulative energy.

Techniques for analyzing signals can inspire methods for understanding probability distributions.

Economics - cumulative wealth distribution

CDFs model how wealth accumulates across a population, similar to probability accumulation.

Seeing CDFs in economics shows how cumulative functions describe real-world distributions beyond math.

Common Pitfalls

#1 Confusing PDF value with probability at a point

Wrong approach:

prob = norm.pdf(0) # Treating this as probability of exactly 0

Correct approach:

prob = norm.cdf(0.0001) - norm.cdf(-0.0001) # Probability in a small range around 0

Root cause:

Misunderstanding that continuous probabilities are areas, not point values.

#2 Manually integrating PDF to get CDF without tools

Wrong approach:

cdf = integrate.quad(norm.pdf, -np.inf, x) # Complex and error-prone

Correct approach:

cdf = norm.cdf(x) # Use built-in scipy function

Root cause:

Not knowing scipy provides direct CDF functions leads to unnecessary complexity.

#3 Assuming PDF must be less than or equal to 1

Wrong approach:

if norm.pdf(x) > 1:
print('Invalid PDF') # Wrong rejection

Correct approach:

print('PDF value:', norm.pdf(x)) # Accept values >1 for narrow distributions

Root cause:

Confusing probability range with density scale.

Key Takeaways

Probability density functions (PDF) describe how probability is spread over continuous values as a density, not direct probabilities.

Cumulative distribution functions (CDF) accumulate probabilities up to a point, making it easy to find chances below thresholds.

scipy provides built-in, reliable functions to compute PDF and CDF for many distributions, avoiding manual integration.

PDF values can exceed 1 for narrow distributions; what matters is the total area under the PDF curve equals 1.

Understanding the difference between density and probability prevents common mistakes and deepens statistical intuition.

Practice

1. What does the Probability Density Function (PDF) represent in a data distribution?

easy

A. The average value of the data

B. The likelihood of a value occurring at a specific point

C. The total probability up to a certain value

D. The maximum value in the data

Solution

1. Step 1: Understand PDF meaning

   The PDF shows how dense or likely values are at specific points in a distribution.

2. Step 2: Differentiate from CDF

   The CDF shows total probability up to a point, not the likelihood at a point.

3. Final Answer:

   The likelihood of a value occurring at a specific point -> Option B

4. Quick Check:

   PDF = Likelihood at point [OK]

Hint: PDF shows likelihood at exact points, not cumulative [OK]

Common Mistakes:

• Confusing PDF with CDF
• Thinking PDF gives total probability
• Assuming PDF gives average or max value

2. Which of the following is the correct way to import the cumulative distribution function (CDF) for a normal distribution from scipy.stats?

easy

A. import norm from scipy.stats; norm.cdf(x)

B. import scipy.stats.norm as norm; norm.cdf(x)

C. from scipy import norm; norm.cdf(x)

D. from scipy.stats import norm; norm.cdf(x)

Solution

1. Step 1: Recall scipy.stats import syntax

   The correct import for normal distribution functions is from scipy.stats import norm.

2. Step 2: Use norm.cdf(x) to get CDF

   After import, norm.cdf(x) computes the cumulative distribution function at x.

3. Final Answer:

   from scipy.stats import norm; norm.cdf(x) -> Option D

4. Quick Check:

   Correct import and usage = from scipy.stats import norm; norm.cdf(x) [OK]

Hint: Use 'from scipy.stats import norm' for normal distribution functions [OK]

Common Mistakes:

• Wrong import syntax causing errors
• Using 'import scipy.stats.norm as norm' which is invalid
• Trying to import norm directly from scipy

3. What is the output of the following code snippet?

from scipy.stats import norm
result = norm.pdf(0, loc=0, scale=1)
print(round(result, 3))

medium

A. 0.399

B. 0.500

C. 1.000

D. 0.000

Solution

1. Step 1: Understand norm.pdf at mean

   For a normal distribution with mean 0 and std 1, PDF at 0 is 1/sqrt(2*pi) ≈ 0.3989.

2. Step 2: Round the result

   Rounding 0.3989 to 3 decimals gives 0.399.

3. Final Answer:

   0.399 -> Option A

4. Quick Check:

   PDF at mean = 0.399 [OK]

Hint: PDF at mean of standard normal ≈ 0.399 [OK]

Common Mistakes:

• Confusing PDF with CDF value
• Rounding incorrectly
• Assuming PDF at mean is 0.5

4. Identify the error in this code that tries to calculate the CDF of a normal distribution at x=1:

import scipy.stats as stats
cdf_value = stats.norm.pdf(1, loc=0, scale=1)
print(cdf_value)

medium

A. Using pdf() instead of cdf() to get cumulative probability

B. Missing import statement for norm

C. Incorrect parameters for norm.pdf

D. Syntax error in print statement

Solution

1. Step 1: Check function used for CDF calculation

   The code uses norm.pdf which gives probability density, not cumulative probability.

2. Step 2: Correct function for cumulative probability

   To get cumulative probability, norm.cdf should be used instead of norm.pdf.

3. Final Answer:

   Using pdf() instead of cdf() to get cumulative probability -> Option A

4. Quick Check:

   Use cdf() for cumulative probability [OK]

Hint: Use cdf() for cumulative, pdf() for density [OK]

Common Mistakes:

• Mixing pdf and cdf functions
• Assuming pdf returns cumulative values
• Ignoring function purpose

5. You have a dataset of exam scores that roughly follow a normal distribution with mean 70 and standard deviation 10. Using scipy, how would you find the probability that a student scored less than 85?

hard

A. Use norm.cdf(70, loc=85, scale=10) to get cumulative probability

B. Use norm.pdf(85, loc=70, scale=10) to get cumulative probability

C. Use norm.cdf(85, loc=70, scale=10) to get cumulative probability

D. Use norm.pdf(70, loc=85, scale=10) to get cumulative probability

Solution

1. Step 1: Identify correct function for cumulative probability

   To find probability of scoring less than 85, use the cumulative distribution function norm.cdf.

2. Step 2: Use correct parameters for mean and std

   Set loc=70 (mean) and scale=10 (std deviation), and evaluate at 85.

3. Final Answer:

   Use norm.cdf(85, loc=70, scale=10) to get cumulative probability -> Option C

4. Quick Check:

   norm.cdf(value, loc=mean, scale=std) = cumulative probability [OK]

Hint: CDF with loc=mean, scale=std, value=threshold [OK]

Common Mistakes:

• Using pdf instead of cdf for cumulative probability
• Swapping mean and value parameters
• Confusing scale with loc