Probability Theory

#folder

Probability Space

Definition

A probability space or probability triple () is a mathematical construct that provides a formal model for a random experiment. A probability space contains three elements:

  • Sample space (): the set of all possible outcomes of an experiment
  • Event space (): the sigma-algebra of subsets of
    • contains subsets of , while satisfying the properties of a -algebra. Each of these subsets is called an event.
  • Probability function (): assigns each event in the event space a probability, which is a number between 0 and 1
Example

Suppose an experiment consists of extracting a ball from a ball containing two balls (red () and blue ).
The sample space is

The event space is

  • : either a blue ball or red ball is extracted
  • : nothing happens (complement of )
  • : a red ball is extracted
  • : a blue ball is extracted (complement of )

And the probability measure of on is:

Example

A fair coin is tossed twice.

The sample space is
An example event is , which is the event "as least one head"

Theoretical Probability

See Kolmogorov Axioms

Let () be a probability space. That is, is the sample space, is the event space, and is a probability function.

Equation for Probability

Classical Definition

Definition

The theoretical probability of an event is

where all the outcomes of are equally likely

(see cardinality)

Alternate notation: .

Example

Roll a 6-sided die twice. Find

solution

So

Example

Leibniz's mistake: if we roll two fair dice, what sum is more likely, 11 or 12?

solution
so .
so

But Leibniz thought that the order mattered and that , which is wrong.

Problems with the classical definition:

Relative Frequency Definition

Definition

the long term relative frequency of an event

Problems:

Complementary Events

$

complement

Complement

Definition - Without explicit mention of the universe

is the set of elements not in

Definition - With explicit mention of the universe

is the set of all elements in but not in

Equivalent to

Alternate notation: , ,

Axiom - Absolute Complement

Law

De Morgan's laws:

in fact, this is how De Morgan's Laws are originally defined, and the complement of a boolean is actually defined by sets too.

Definition

A complement of an event is the opposite event, like the complement. The complement of event is written as .

and

Finding Probability with Sets

AND, Intersection

Intersection

Definition

is the set of all elements belonging to both and

Axiom - Set Intersection

Properties - Set Operators

  1. Commutativity:
  2. Associativity:
  3. Distributivity:

Definition

Given two events and , the probability of AND occurring is

Mutually Exclusive Events

$

disjoint

Disjoint

Definition

Two sets are disjoint or mutually exclusive if and only if

In other words, they share no common elements, so their intersection is empty.

Pairwise Disjoint

Definition

Given arbitrary sets , we say these sets are pairwise disjoint if all of them are disjoint with each other. That is, none of them share common elements.

Definition

If two events and are disjoint, then they are mutually exclusive events.

Theorem

Two events and if and only if

OR, Union

Union

Definition

is the set of all elements belonging to either or

(see Set Builder Notation, logical operators)

Axiom - Set Union


(see If and Only If)

Properties

  1. Commutativity:
  2. Associativity:
  3. Distributivity:

Additive Principle for the Union of Sets

Principle

For all additive set functions , and sets and (not necessarily disjoint), we have:

Definition

Given two events and , the probability of OR occurring is

Additive Principle for Unions of Two Sets

Additive Principle for the Union of Sets

Principle

For all additive set functions , and sets and (not necessarily disjoint), we have:

Extending this definition to out probability and cardinality functions:

Definition

and

Intuition

The intersection must be subtracted to avoid double counting, since A and B may overlap, and we only want to count the overlap once.

Example

Determine the the probability of drawing a card that is red or an ace

Example

Two fair die are rolled. What is the probability that at least 1 is a six?
solution

Definition - For 3 Events

[!example]
Let . Find the number of sets where is a set of elements in that are coprime to .

solution
We can find the probability of the complement. Prime factor: .

Example

A random card hand is dealt from a deck of 52 cards. Find the probability that at least one suit is not in the dealt hand.

solution

  • : no clubs
  • : no hearts
  • : no diamonds
  • : no spades

Since all the values will be the same, we can write: