Raised Fist0

Bayes’ Theorem

Start learning this pattern below

Jump into concepts and practice - no test required

or
Recommended
Test this pattern10 questions across easy, medium, and hard to know if this pattern is strong

Introduction

Bayes’ Theorem helps us revise probabilities when new evidence is obtained. It is widely used in real-world applications such as medical diagnosis, machine learning, and decision-making where we need to determine the probability of an event after observing related data.

Pattern: Bayes’ Theorem

Pattern: Bayes’ Theorem

Bayes’ Theorem updates the probability of an event A after observing event B.

Formula:
P(A | B) = [P(B | A) × P(A)] / [P(B | A)P(A) + P(B | A′)P(A′)]

Step-by-Step Example

Question

Two machines, M₁ and M₂, produce bulbs. M₁ produces 40% of the bulbs with a defect rate of 2%. M₂ produces 60% of the bulbs with a defect rate of 3%. If a bulb is defective, what is the probability it was made by M₁?

Solution

  1. Step 1: Define the known probabilities

    P(M₁) = 0.4, P(M₂) = 0.6
    P(D|M₁) = 0.02, P(D|M₂) = 0.03

  2. Step 2: Apply Bayes’ Theorem

    P(M₁|D) = [P(D|M₁) × P(M₁)] / [P(D|M₁)P(M₁) + P(D|M₂)P(M₂)]

  3. Step 3: Substitute and simplify

    = (0.02 × 0.4) / [(0.02 × 0.4) + (0.03 × 0.6)]
    = 0.008 / (0.008 + 0.018) = 0.008 / 0.026 = 0.3077

  4. Final Answer:

    30.77% (≈ 0.31)

  5. Quick Check:

    M₁ contributes 0.008 defective bulbs out of total defective rate 0.026 → 0.008/0.026 = 0.3077 ✅

Quick Variations

1. Medical Test: Given a positive test, find probability of actually having the disease.

2. Quality Control: Given a defective product, find which machine produced it.

3. Email Classification: Given that an email contains certain words, find probability it is spam.

Trick to Always Use

  • Step 1: Identify all hypotheses (A, A′) with their priors.
  • Step 2: Find P(B|A) and P(B|A′) - the likelihoods of evidence under each hypothesis.
  • Step 3: Apply Bayes’ formula and divide the numerator by total evidence probability.
  • Step 4: Convert decimal to percentage for easy interpretation.

Summary

In the Bayes’ Theorem pattern:

  • It helps in finding reverse probabilities like P(A|B).
  • Formula used: P(A|B) = [P(B|A) × P(A)] / [P(B|A)P(A) + P(B|A′)P(A′)].
  • Always identify priors (P(A)), likelihoods (P(B|A)), and normalize using the total probability of B.
  • Useful in cases where you want to update your belief after seeing new evidence.
  • Quick Check: The probabilities in denominator should sum to the total probability of the evidence.

Practice

(1/5)
1. Box A contains 3 red and 2 blue balls. Box B contains 1 red and 4 blue balls. A box is chosen at random and one ball drawn; it is red. What is the probability the ball came from Box A?
easy
A. 3/4
B. 1/4
C. 1/2
D. 2/3

Solution

  1. Step 1: Write priors and likelihoods

    P(A)=1/2, P(B)=1/2. P(red|A)=3/5, P(red|B)=1/5.

  2. Step 2: Apply Bayes' rule

    P(A|red) = [P(red|A)P(A)] / [P(red|A)P(A) + P(red|B)P(B)].

  3. Step 3: Substitute

    =(3/5×1/2) / [(3/5×1/2) + (1/5×1/2)] = (3/10) / (4/10) = 3/4.

  4. Final Answer:

    3/4 → Option A.

  5. Quick Check:

    Box A is more likely to produce red (3/5 vs 1/5), so posterior > 1/2 → 3/4 ✅
Hint: Posterior ∝ prior × likelihood; normalize across boxes.
Common Mistakes: Forgetting to weight by the prior probability of choosing each box.
2. A disease has prevalence 1%. A test has 99% sensitivity (P(+|disease)=0.99) and 5% false positive rate (P(+|no disease)=0.05). If a randomly chosen person tests positive, what is the probability they actually have the disease?
easy
A. 1/20
B. 1/6
C. 1/10
D. 1/4

Solution

  1. Step 1: Write priors and likelihoods

    P(D)=0.01, P(not D)=0.99. P(+|D)=0.99, P(+|not D)=0.05.

  2. Step 2: Bayes' formula

    P(D|+) = P(+|D)P(D) / [P(+|D)P(D) + P(+|not D)P(not D)].

  3. Step 3: Substitute and compute

    = (0.99×0.01) / (0.99×0.01 + 0.05×0.99) = 0.0099 / (0.0099 + 0.0495) = 0.0099 / 0.0594 = 1/6 (≈0.1667).

  4. Final Answer:

    1/6 → Option B.

  5. Quick Check:

    Low prevalence makes false positives dominate despite high sensitivity → posterior ~16.7% ✅
Hint: Compute numerator = sensitivity×prevalence; denominator = numerator + falsePositiveRate×(1-prevalence).
Common Mistakes: Ignoring prevalence and treating sensitivity as posterior directly.
3. Machine M1 produces 40% of items with 2% defective rate; M2 produces 60% with 3% defective rate. A randomly chosen item is defective. What is the probability it came from M1?
easy
A. 0.5
B. 0.25
C. 0.3077
D. 0.6923

Solution

  1. Step 1: Write priors and likelihoods

    P(M1)=0.40, P(M2)=0.60. P(D|M1)=0.02, P(D|M2)=0.03.

  2. Step 2: Bayes' formula

    P(M1|D) = P(D|M1)P(M1) / [P(D|M1)P(M1) + P(D|M2)P(M2)].

  3. Step 3: Substitute

    =(0.02×0.40) / (0.02×0.40 + 0.03×0.60) = 0.008 / (0.008 + 0.018) = 0.008 / 0.026 = 0.3077.

  4. Final Answer:

    0.3077 → Option C.

  5. Quick Check:

    M1 contributes 0.008 of defects vs total 0.026 → 0.008/0.026 ≈ 30.77% ✅
Hint: Posterior = (likelihood×prior) / sum(likelihood×prior) across sources.
Common Mistakes: Using raw defect rates without weighting by production share.
4. Urn U1 has 2 white and 3 black balls; urn U2 has 4 white and 1 black. An urn is chosen with P(U1)=1/3 and P(U2)=2/3. A drawn ball is white. What is the probability it came from U1?
medium
A. 1/5
B. 2/5
C. 3/5
D. 4/5

Solution

  1. Step 1: Priors and likelihoods

    P(U1)=1/3, P(U2)=2/3. P(white|U1)=2/5, P(white|U2)=4/5.

  2. Step 2: Bayes' rule

    P(U1|white) = [P(white|U1)P(U1)] / [P(white|U1)P(U1) + P(white|U2)P(U2)].

  3. Step 3: Compute

    =(2/5×1/3) / [(2/5×1/3) + (4/5×2/3)] = (2/15) / (2/15 + 8/15) = (2/15) / (10/15) = 1/5.

  4. Final Answer:

    1/5 → Option A.

  5. Quick Check:

    U2 is more likely a priori and also likelier to give white, so posterior for U1 is small → 1/5 ✅
Hint: Compute contributions to evidence from each urn, then take ratio for the target urn.
Common Mistakes: Failing to multiply likelihood by the correct prior for each urn.
5. Three machines M1, M2, M3 produce 20%, 30%, 50% of items with defect rates 1%, 2%, 4% respectively. If an item is defective, what is the probability it came from M3?
medium
A. 1/3
B. 5/14
C. 2/7
D. 5/7

Solution

  1. Step 1: Compute contributions

    Numerators: M1 → 0.01×0.20=0.002; M2 → 0.02×0.30=0.006; M3 → 0.04×0.50=0.020.

  2. Step 2: Total defective probability

    Sum = 0.002 + 0.006 + 0.020 = 0.028.

  3. Step 3: Bayes' posterior for M3

    P(M3|defect) = 0.020 / 0.028 = 20/28 = 5/7 ≈ 0.7143.

  4. Final Answer:

    5/7 → Option D.

  5. Quick Check:

    M3 contributes the largest share of defects (0.02 of 0.028) → ~71.43% ✅
Hint: For multiple sources, compute likelihood×prior for each, then normalize by their sum.
Common Mistakes: Neglecting any machine's contribution when calculating the total defective probability.