Bayes Theorem

TL;DR

• Combines prior prevalence and the likelihood of evidence to produce a revised probability for an event.
• Explains why a highly accurate test can still yield a low probability of disease when the condition is rare (example: 8.3% posterior probability).
• Used to update beliefs as new information becomes available.

Definition

Bayes’ Theorem is a mathematical formula that calculates the probability of an event given new evidence by combining the probability of the evidence given the event with the prior probability of the event, divided by the probability of the evidence:

$$P(\text{event} \mid \text{evidence}) = \frac{P(\text{evidence} \mid \text{event}) \cdot P(\text{event})}{P(\text{evidence})}$$

Explanation

• The theorem formalizes how to update beliefs: prior beliefs (the initial probability of an event) are adjusted in light of new evidence.
• The numerator, (P(\text{evidence} \mid \text{event}) \cdot P(\text{event})), combines how likely the evidence is if the event is true with the prior probability of the event.
• The denominator, (P(\text{evidence})), accounts for how common the evidence is overall (including when the event is false).
• Applying the theorem yields a posterior probability, the updated probability of the event after observing the evidence.

Examples

Rare disease and diagnostic test

• Scenario: A disease affects 1% of the population and the test for the disease is 90% accurate. A person tests positive.
• Using Bayes’ Theorem, the probability that the person actually has the disease given the positive test is computed from the prior (1%) and the test accuracy (90%).
• This gives a final probability of 8.3% that the person actually has the disease, even though they tested positive.

Medical diagnosis with symptoms

• When a patient presents with symptoms (for example, fever, headache, and rash) that are consistent with a disease such as the flu, a doctor can use Bayes’ Theorem to calculate the probability the patient has the flu.
• The calculation takes into account the prevalence of the flu in the population, the accuracy of diagnostic tests, and the patient’s other symptoms to produce a more informed diagnostic probability.

Use cases

• Statistics: calculate probabilities of outcomes based on sample data and prior information.
• Machine learning: compute the likelihood that a data point belongs to a class using class characteristics and overall data.

Notes or pitfalls

• A positive test result does not necessarily imply a high probability of disease when the disease prevalence is low; posterior probability can be much lower than test accuracy alone suggests.
• Accurate updating requires consideration of both prior prevalence and the accuracy (likelihood) of the evidence.

Related terms

• Prior probability (prior beliefs)
• Posterior probability (probability of an event given evidence)
• Statistics
• Machine learning