Statistical Modeling

#folder

Setup:

Parameter: is an unknown attribute of the population
A statistical model:
todo

[!example]
Suppose that we toss a coin 100 times and we are interested in finding .

then we have and ...

[!example]
where the starting salary of an SE grad from UW

Suppose we look at the data and it appears to be normal, then

with and unknown.

Notes:

  1. Attributes of the population that you are interested in are typically the parameters of your model
  2. Finding a model is an empirical question, not theoretical

Estimation

Problem:
is a sequence of i.i.d. r.v.'s with p.m.f. where is an unknown parameter.

Data: is the outcomes of the r.v.'s that we observe
goal: construct (the estimate of )

Notes:

  1. - unknown constant -> will never be known, otherwise we don't need to estimate it

[!example]
A coin is tossed 100 times with . Suppose that we know the coin is biased:

We do the experiment and observe 60 heads

Suppose , then
If , then , which is the larger value

Idea: likelyhood ->

where is a likelyhood function

Setup:
what are we given:

What is the model?

What is the objective?
To estimate with where is a function of our data.

Likelyhood Function

Definition

If where are i.i.d. RVs with observations , then

Maximum Likelihood Estimate (MLE)

Definition

is the MLE if maximizes

The value of that is most likely to have generated your data

[!properties]

  1. Consistency: As ,
  2. Efficiency
  3. Invariance: if is the MLE of , then is the MLE of

Binomial MLE

Suppose with observed successes. Then what is ?

solution
suppose and . Intuitively,

Likelyhood: (use optimization to maximize)
Log likelyhood:

so

now set:

which is exactly .

todo organize, understand

Poisson MLE

Suppose with observations/data of . What is the MLE of ?

solution
todo calculus

Continuous MLEs

Model:
todo finish

Exponential MLE

Suppose with observations/data of . What is the MLE of ?

Suppose . Find the MLE for the median of .

  1. Find median, m

F(m) & = \frac{1}{2}, \quad \text{where } F(m) = P(Y \le m) \
1 - e^{-\lambda m} & = \frac{1}{2} \
m & = -\frac{1}{\lambda} \ln\left( \frac{1}{2} \right)
\end

You can't use 'macro parameter character #' in math mode2. Since $\hat{\lambda} = \dfrac{1}{\oline{y}}$, $\hat{m} = - \oline{y} \ln(\frac{1}{2})$. #### Normal MLE ### Relative Likelihood Function > [!definition] > $$R(\theta) = \frac{L(\theta)}{L(\hat{\theta})}

is the likelihood function and is the MLE of

Remember that
For large enough , is extremely small. is a way to "normalize" (standardize?) :

since is maximized.

[!example]
Suppose a coin is tossed 200 times and we observe heads and

We have and .

This is the relative likelihood of being 0.7 given that (i.e what we observed)

wtf is bro waffling about I don't get it

means that is twice less likely less to have occurred than