Probability Distributions

is a random variable (r.v.) if it is a function that maps each outcome of a random experiment to an element in .

When we write:

We are saying "the distribution of looks like , and we can use the distribution to approximate ". Hence more or less becomes an alias of the distribution function.

For example, if we ask for , we are really asking for .

The set of all possible outputs of a random variable is called the support or sample space (i.e the range of the function/random variable, which is also the domain of the probability function).

Random variables and are independent iff :

"comma pretty much means and"

(see universally quantifier)

Suppose two dice are rolled and we define random variables:

Are and independent?

solution
Counterexample:

If happens then there's no way could've happened, and vice versa.

Since , and are dependent.

The c.d.f of is defined as .

In the case that is discrete, the c.d.f. is defined as:

In the case that is continuous, the c.d.f. is defined as:

(see summation notation, integral, improper integral, Riemann Sum)

Compare this to the definition of a p.m.f: . The c.d.f. is the accumulation of those values up to a certain point.

1. (increasing)
2. (monotone)
3. (see limit)
4. is a step function in the discrete case
5. Right-continuous

In the continuous case:

1. Also left continuous

Given the following p.m.f.:

Find the c.d.f

solution

Four die are rolled simultaneously. Suppose the random variable . Find the p.m.f of .

solution
As an example, the calculation for is not nice. But are known as IID (independent and identically distributed) variables, and we can make a connection to c.d.f.

Example calculations:

A probability distribution is a table, formula, or graph that provides the probabilities of a random variable assuming any of its possible values

The probability distribution where is the sum of two dice:

The theoretical probability distribution of the sum of two dice:

This is an example of a discrete probability distribution. If we were measuring the distribution of say, height, we would have a continuous probability distribution.

The excepted value of a random variable is the mean value after many repetitions if an experiment.

More precisely:

Let be a discrete random variable with a p.m.f. , then

Let be a continuous random variable with p.d.f. , then

That is, the expected value of a discrete random variable, , is the sum of the terms of the form for all possible values of .

(see Summation and Product Notation)

Often you may see the following "notation" of the formula, which may help with understanding:

"Expected value" is a bit of a misnomer. It's more like "weighted average". The expectation isn't that you will actually get the expected value, and often it's impossible.

Example:

but isn't even an option in our sample space.

• Non-negativity:
  • If then
• Non-degeneracy
• Linearity
• Monotonicity
• Deterministic
• Constants
  • For all real constants ,
• Non-multiplicity
  • If and are independent, then
  • If and are dependent, then generally , although there are special cases where the equality holds

Calculate the expected value, , for the sum of two dice

solution

Suppose you want to select a committee using three people. The group which the committee members can be selected consists of 4 Waterloo residents and 3 Kitchener residents.

1. Create a probability distribution for the number of Kitchener residents on the committee

2. Calculate the probability that at least one Kitchener resident is on the committee

3. Calculate the expected number of Kitchener residents on the committee

You are developing a platform to analyze the performance of your friend’s website selling Goose-themed merchandise.
Suppose that you are working on an algorithm that will highlight two different products to a customer, product and product . The probabilities of a customer buying product and product are and , respectively. Your friend will earn $10 for selling product A and $20 for selling product B.

You are trying to decide which product to display to the customer first. If a customer does not buy the first product highlighted to them, then they will not buy the second product. If you can choose which order to highlight each product, justify a strategy to display the products for maximizing your friend’s profits.

solution
Let be the profit if is highlighted, and be the profit if is highlighted.

So it's more profitable to display product first. This may seem counterintuitive if you don't consider that the customer may buy the other product if a sale is complete, but will not buy the other product if it is not.

Given a discrete random variable and p.m.f. , the expected value of is:

Given a continuous random variable and p.d.f. , the expected value of is:

Why not ? Because the random variable is still weighed by its frequency, which is dictated by , not .

A fair coin is tossed until a head appears. The casino will pay:

• $2 if a head appears on the first toss
• $4 if a head appears on the second toss
• $8 if a head appears on the third toss

How much are you willing to pay to play the game? What is the expected payoff?

solution
Let be a random variable for the amount of money won.

The paradox: the expected payoff is , so you're willing to pay any amount of money to play, right? Certainly not. No sane person would look at the problem that way.

If is a random variable, then it is said to be memoryless if for any integers and ,

(see conditional event)

Suppose ,

As the name suggests, we constantly "forget" the current state of the system. That is, probabilities are not influenced by the history of the process.

With Memory
Suppose is a random variable representing the lifetime of a car engine, representing the "number of km driven until the engine breaks down". An engine that already has driven 500 000 km is will have a much lower than would a second engine which has only been driven 1000 km.

Without Memory
Suppose there's a long hallway, lined on one wall with thousands of safes. The dial on each safe has 500 positions, and each has been randomly assigned an opening position. Imagine an eccentric person walks down the hallway, stopping once at each safe to make a single random attempt to open it.

We define random variable as the "lifetime of the search", representing the number of attempts the person must make until the successfully open a safe.

In this case, , regardless of how many attempts have already been made. Each attempt has a 1/500 chance of succeeding, but with each new failure, this person makes no progress. Even if the cracker failed 499 times (or even 4 999 times, 10 000 times, etc (St. Petersburg paradox)), we expect to wait 500 more attempts until we observe the next success.

If this person instead focused on one safe, they would need at most 500 attempts to open the safe (and we would expect it to take 250 attempts), since they would remember.