Bayes Theorem: Explained with Examples, Notes and FAQs

Bayes theorem reverses conditional probabilities: it computes P(Ei | A), the probability that route Ei was the true one, given that A was observed. The formula is P(Ei | A) = P(Ei) × P(A | Ei) / P(A), where the denominator P(A) is expanded by the total probability theorem.

This matters because life runs backwards from the data we have. A doctor sees a positive test and needs P(disease | positive), not P(positive | disease); a spam filter sees words and needs P(spam | words). Bayes theorem is the only correct bridge between the direction we can measure and the direction we need. It is the mathematical heart of medical testing, machine learning classifiers, and scientific reasoning: start with priors, multiply by how well each hypothesis explains the evidence, and renormalise.

Key Concepts

• Prior P(Ei): belief about hypothesis Ei before seeing the evidence A.
• Likelihood P(A | Ei): how probable the observed evidence is under hypothesis Ei.
• Evidence P(A): total probability of the observation, Σ P(Ej) × P(A | Ej), the normalising constant.
• Posterior P(Ei | A): updated belief about Ei after observing A; proportional to prior × likelihood.
• Partition requirement: the hypotheses E1 … En must be mutually exclusive and exhaustive.

Formulas

Bayes theorem: P(Ei | A) = P(Ei) × P(A | Ei) / Σ P(Ej) × P(A | Ej), summed over all j. Two-hypothesis version: P(E | A) = P(E) × P(A | E) / [P(E) × P(A | E) + P(E′) × P(A | E′)]. The numerator is prior × likelihood for the hypothesis of interest; the denominator is the total probability of the evidence. It assumes P(A) > 0.

Worked Example

Urn I has 3 red and 2 white balls, Urn II has 2 red and 4 white balls, and a fair coin picks the urn. A red ball is drawn. What is the probability it came from Urn I? We need P(E1 | red). The prior is P(E1) = 1/2 and the likelihood P(red | E1) = 3/5, so the numerator is (1/2) × (3/5) = 3/10. The denominator is P(red) = 7/15 from total probability. Therefore P(E1 | red) = (3/10) / (7/15) = (3/10) × (15/7) = 45/70 = 9/14, about 0.643. Observing red raised the chance of Urn I from the prior 0.5 to the posterior 0.643, because Urn I is redder.

Common Misconceptions

• Posterior equals likelihood: P(Ei | A) is not P(A | Ei). The prior and the normalising denominator matter; ignoring them is the base-rate fallacy behind many false alarms.
• The denominator can be skipped: Without dividing by P(A), the numbers do not sum to 1 across hypotheses and are not probabilities at all.
• Bayes needs equally likely priors: No; priors can be any valid probabilities. The fair-coin 1/2 above was just this example's setup.

Real-World Applications

Bayes theorem powers medical screening, converting a positive result into an actual disease probability using test accuracy and disease prevalence. Email spam filters, speech recognition and many classifiers are naive Bayes models. In science it formalises how experiments shift belief between competing theories, and in law it correctly combines prior suspicion with forensic evidence.

Bayes theorem updates beliefs via P(Ei | A) = P(Ei) × P(A | Ei) / P(A): prior times likelihood, normalised by the total probability of the evidence. It converts measurable likelihoods into the posteriors we actually need, from medical tests to spam filters. Correct use demands genuine priors, a real partition of hypotheses, and the normalising denominator that the base-rate fallacy forgets.