Law of Total Probability and Baye's Theorem

Theorem

Let , where for (that is, every is disjoint). Then for any event :

(see conditional event, summation notation, empty set)

Example

You have one fair coin and a biased coin that lands on heads with a probability of . If you pick a random coin and toss it three times, find the probability that all three tosses are heads.

solution
We know:

Baye's Theorem

Theorem

Let be a partition of , and be any event, then

Proof

Note - Terminology

  • : prior probabilities
  • : posterior probability

Example

You have one fair coin and a biased coin that lands on heads with a probability of . A coin is chosen at random and tossed three times. If we observe three heads in a row, what is the probability that the fair coin was chosen?

= the fair coin was chosen
= the biased coin was chosen
= 3 heads observed in 3 tosses

solution

Bayes rule: find

Baseline Fallacy

Question

Lets say you have 2 groups:

  1. Group 1: 10 people, 90% of people can play piano
  2. Group 2: 1000 people, 5% can play piano

If you meet someone who can play piano, which group are they more likely to be from?

answer
Our immediate answer may be group 1, since 90% of people play piano. But think again:

  1. Group 1: people can play piano
  2. Group 2: people can play piano

So it's actually almost 6x more likely that this person is from group 2, since the absolute numbers in terms of population is so high.

Example

A person is getting tested for a disease that affects 1% of the population.

A test is 95% accurate in true positivity (sensitivity)
A test is 95% accurate in true negativity (specificity)

If a person tests positive, then what is the probability that they have the disease?

solution
Let:

  • = has the disease,
  • = doesn't have the disease,
  • = test is positive
  • = test is negative


What is (that is, given someone tests positive, how likely are they to have the disease)?
Intuitively, we would say 95%, but this isn't correct.

So If you test positive, there's only a 16% chance you actually have the disease.

Explanation:
Suppose we have 100 000 people in population. Then 1000 people have the disease (1%).
Hence we have 99 000 without the disease, and 1000 with the disease.

  • 99000 without the disease
    • Test positive: 4 950
    • Test negative: 94 050
  • 1000 with the disease
    • Test positive: 950
    • Test negative: 50

Far more people WITHOUT the disease will test positive than those WITH the disease, which is why the percentage is so low.