0
0

Bayes’ Theorem

Introduction

Bayes’ Theorem helps us revise probabilities when new evidence is obtained. It is widely used in real-world applications such as medical diagnosis, machine learning, and decision-making where we need to determine the probability of an event after observing related data.

Pattern: Bayes’ Theorem

Pattern

Bayes’ Theorem updates the probability of an event A after observing event B.

Formula:
P(A | B) = [P(B | A) × P(A)] / [P(B | A)P(A) + P(B | A′)P(A′)]

Step-by-Step Example

Question

Two machines, M₁ and M₂, produce bulbs. M₁ produces 40% of the bulbs with a defect rate of 2%. M₂ produces 60% of the bulbs with a defect rate of 3%. If a bulb is defective, what is the probability it was made by M₁?

Solution

  1. Step 1: Define the known probabilities

    P(M₁) = 0.4, P(M₂) = 0.6
    P(D|M₁) = 0.02, P(D|M₂) = 0.03

  2. Step 2: Apply Bayes’ Theorem

    P(M₁|D) = [P(D|M₁) × P(M₁)] / [P(D|M₁)P(M₁) + P(D|M₂)P(M₂)]

  3. Step 3: Substitute and simplify

    = (0.02 × 0.4) / [(0.02 × 0.4) + (0.03 × 0.6)]
    = 0.008 / (0.008 + 0.018) = 0.008 / 0.026 = 0.3077

  4. Final Answer:

    30.77% (≈ 0.31)

  5. Quick Check:

    M₁ contributes 0.008 defective bulbs out of total defective rate 0.026 → 0.008/0.026 = 0.3077 ✅

Quick Variations

1. Medical Test: Given a positive test, find probability of actually having the disease.

2. Quality Control: Given a defective product, find which machine produced it.

3. Email Classification: Given that an email contains certain words, find probability it is spam.

Trick to Always Use

  • Step 1: Identify all hypotheses (A, A′) with their priors.
  • Step 2: Find P(B|A) and P(B|A′) - the likelihoods of evidence under each hypothesis.
  • Step 3: Apply Bayes’ formula and divide the numerator by total evidence probability.
  • Step 4: Convert decimal to percentage for easy interpretation.

Summary

Summary

In the Bayes’ Theorem pattern:

  • It helps in finding reverse probabilities like P(A|B).
  • Formula used: P(A|B) = [P(B|A) × P(A)] / [P(B|A)P(A) + P(B|A′)P(A′)].
  • Always identify priors (P(A)), likelihoods (P(B|A)), and normalize using the total probability of B.
  • Useful in cases where you want to update your belief after seeing new evidence.
  • Quick Check: The probabilities in denominator should sum to the total probability of the evidence.

Practice

(1/5)
1. Box A contains 3 red and 2 blue balls. Box B contains 1 red and 4 blue balls. A box is chosen at random and one ball drawn; it is red. What is the probability the ball came from Box A?
easy
A. 3/4
B. 1/4
C. 1/2
D. 2/3

Solution

  1. Step 1: Write priors and likelihoods

    P(A)=1/2, P(B)=1/2. P(red|A)=3/5, P(red|B)=1/5.

  2. Step 2: Apply Bayes' rule

    P(A|red) = [P(red|A)P(A)] / [P(red|A)P(A) + P(red|B)P(B)].

  3. Step 3: Substitute

    =(3/5×1/2) / [(3/5×1/2) + (1/5×1/2)] = (3/10) / (4/10) = 3/4.

  4. Final Answer:

    3/4 → Option A.

  5. Quick Check:

    Box A is more likely to produce red (3/5 vs 1/5), so posterior > 1/2 → 3/4 ✅
Hint: Posterior ∝ prior × likelihood; normalize across boxes.
Common Mistakes: Forgetting to weight by the prior probability of choosing each box.
2. A disease has prevalence 1%. A test has 99% sensitivity (P(+|disease)=0.99) and 5% false positive rate (P(+|no disease)=0.05). If a randomly chosen person tests positive, what is the probability they actually have the disease?
easy
A. 1/20
B. 1/6
C. 1/10
D. 1/4

Solution

  1. Step 1: Write priors and likelihoods

    P(D)=0.01, P(not D)=0.99. P(+|D)=0.99, P(+|not D)=0.05.

  2. Step 2: Bayes' formula

    P(D|+) = P(+|D)P(D) / [P(+|D)P(D) + P(+|not D)P(not D)].

  3. Step 3: Substitute and compute

    = (0.99×0.01) / (0.99×0.01 + 0.05×0.99) = 0.0099 / (0.0099 + 0.0495) = 0.0099 / 0.0594 = 1/6 (≈0.1667).

  4. Final Answer:

    1/6 → Option B.

  5. Quick Check:

    Low prevalence makes false positives dominate despite high sensitivity → posterior ~16.7% ✅
Hint: Compute numerator = sensitivity×prevalence; denominator = numerator + falsePositiveRate×(1-prevalence).
Common Mistakes: Ignoring prevalence and treating sensitivity as posterior directly.
3. Machine M1 produces 40% of items with 2% defective rate; M2 produces 60% with 3% defective rate. A randomly chosen item is defective. What is the probability it came from M1?
easy
A. 0.5
B. 0.25
C. 0.3077
D. 0.6923

Solution

  1. Step 1: Write priors and likelihoods

    P(M1)=0.40, P(M2)=0.60. P(D|M1)=0.02, P(D|M2)=0.03.

  2. Step 2: Bayes' formula

    P(M1|D) = P(D|M1)P(M1) / [P(D|M1)P(M1) + P(D|M2)P(M2)].

  3. Step 3: Substitute

    =(0.02×0.40) / (0.02×0.40 + 0.03×0.60) = 0.008 / (0.008 + 0.018) = 0.008 / 0.026 = 0.3077.

  4. Final Answer:

    0.3077 → Option C.

  5. Quick Check:

    M1 contributes 0.008 of defects vs total 0.026 → 0.008/0.026 ≈ 30.77% ✅
Hint: Posterior = (likelihood×prior) / sum(likelihood×prior) across sources.
Common Mistakes: Using raw defect rates without weighting by production share.
4. Urn U1 has 2 white and 3 black balls; urn U2 has 4 white and 1 black. An urn is chosen with P(U1)=1/3 and P(U2)=2/3. A drawn ball is white. What is the probability it came from U1?
medium
A. 1/5
B. 2/5
C. 3/5
D. 4/5

Solution

  1. Step 1: Priors and likelihoods

    P(U1)=1/3, P(U2)=2/3. P(white|U1)=2/5, P(white|U2)=4/5.

  2. Step 2: Bayes' rule

    P(U1|white) = [P(white|U1)P(U1)] / [P(white|U1)P(U1) + P(white|U2)P(U2)].

  3. Step 3: Compute

    =(2/5×1/3) / [(2/5×1/3) + (4/5×2/3)] = (2/15) / (2/15 + 8/15) = (2/15) / (10/15) = 1/5.

  4. Final Answer:

    1/5 → Option A.

  5. Quick Check:

    U2 is more likely a priori and also likelier to give white, so posterior for U1 is small → 1/5 ✅
Hint: Compute contributions to evidence from each urn, then take ratio for the target urn.
Common Mistakes: Failing to multiply likelihood by the correct prior for each urn.
5. Three machines M1, M2, M3 produce 20%, 30%, 50% of items with defect rates 1%, 2%, 4% respectively. If an item is defective, what is the probability it came from M3?
medium
A. 1/3
B. 5/14
C. 2/7
D. 5/7

Solution

  1. Step 1: Compute contributions

    Numerators: M1 → 0.01×0.20=0.002; M2 → 0.02×0.30=0.006; M3 → 0.04×0.50=0.020.

  2. Step 2: Total defective probability

    Sum = 0.002 + 0.006 + 0.020 = 0.028.

  3. Step 3: Bayes' posterior for M3

    P(M3|defect) = 0.020 / 0.028 = 20/28 = 5/7 ≈ 0.7143.

  4. Final Answer:

    5/7 → Option D.

  5. Quick Check:

    M3 contributes the largest share of defects (0.02 of 0.028) → ~71.43% ✅
Hint: For multiple sources, compute likelihood×prior for each, then normalize by their sum.
Common Mistakes: Neglecting any machine's contribution when calculating the total defective probability.

Mock Test

Ready for a challenge?

Take a 10-minute AI-powered test with 10 questions (Easy-Medium-Hard mix) and get instant SWOT analysis of your performance!

10 Questions
5 Minutes