ProbabilityDiscussion

October 7, 2010

Statistical Inference (Casella and Berger)

wikipedia:Set (mathematics)

wikipedia:Probability interpretations

http://en.wikipedia.org/wiki/Set_%28mathematics%29

http://en.wikipedia.org/wiki/Probability_interpretations

Set Theory:

Union - combination of two sets

Complement - everything that's not in A

Empty - no elements

Definitions:

experiment - any activity generating observable results

outcome - result of experiment (IMPORTANT TO KEEP STRAIGHT! don't confuse events and outcomes)

trial - single performance of experiment

sample space - set of all possible outcomes

countable/uncountable: - countable = one-to-one correspondence (e.g. 1/n) - uncountable = no one-to-one correspondence can be made -- infinite loop: you can do an infinite loop, but still count it -- flipping a coin: countable; temperature: uncountable

event - any subset of the sample space

Example:

experiment - roll a dice outcome - 1, or 2, or 3, or 4, or 5, or 6 trial - one roll of the dice COUNTABLE sample space - {1,2,3,4,5,6} event - may be {1}, or {1,2,3}, etc...

Operators:

Union, empty set, complement, intersection

Commutative:

Associative:

Distributive:

DeMorgan's Law:

Call a set abnormal if it can be put into itself (otherwise it's normal)

Example: the set of all squares is not itself square, so it is not a member of the set of squares The complimentary set, containing all non-squares, is itself not a square, so is normal

Consider the set of all normal sets Is it normal or abnormal? If it were normal, it would be contained in itself, and would therefore be abnormal If it were abnormal, it would not be contained in itself, and would therefore be normal

You can resolve this using more rigorous set theory...

More Definitions:

Disjoint ("set" term) / mutually exclusive ("probability" term) - if the intersection of two sets is the null set, they are mutually exclusive

Partition - take a group of sets; if the union of these sets is the sample sapce, and they are mutually exclusive, this is a partition

---

Distinction between probability theory that has a physical meaning (and is therefore "contaminated" by intuition) and a more abstract probability theory that doesn't have a corresponding physical meaning

Axiomatic probability theory (Komolgorov)

A probability is a function that follows 3 axioms:

Sample space $ S $

(The domain) $ \sigma $-algebra $ \mathfrak{B} $ (means the set is fully consistent)

Function P -> probability over the domain $ \mathfrak{B} $

1. $ P(A) \geq 0 $ for all $ A \in \mathfrak{B} $

2. $ P(S) = 1 $

3. If $ A \in \mathfrak{B} $ and $ B \in \mathfrak{B} $ are disjoint, then $ P(A \bigcup B) = P(A) + P(B) $

In other words, $ P( \bigcup_{i=1}^{\infty} = \sum_{l=1}^{\infty} P(A_{i}) $

This is a mathematician's viewpoint: a clean definition, as long as we follow these rules, the function is a probability.

What is the probability of the null set?

Create a partition: $ S = {S, \emptyset} $

The probability of the sample space is $ P(S) = 1 $

So $ P(\emptyset) = 1 - P(S) = 0 $

$ P(A) = 1 $

$ P(A^c) = 1-P(A) $

If $ A \subset B $ then $ P(A) \leq P(B) $

The size of the set is directly related to the probability...

Another way to do this is using measure theory (another route, besides rigorous set theory, that leads to probability theory)

[wikipedia:Measure theory]

[wikipedia:Sigma-algebra]

Bonferroni's inequality: $ P(A \bigcap B) \geq P(A) + P(B) - 1 $

Reading Assignment Discussion

Classical Definition of Probability (Laplace, 1812)

"If a random experiment can result in $ N $ mutually exclusive and equally likely outcomes and if $ N_A $ of these outcomes result in the occurrence of the event $ A $, the probability of $ A $ is defined by $ P(A) = P{A} = {N_A \over N} $."

Example: rolling a dice

Event A might be how many times we roll a 1... or how many times we roll a 1 or a 2...

What if one side of the dice is weighted to favor 5? This definition doesn't work... No mathematical proof to show that the outcomes were mutually exclusive and equally likely.

Frequency

This is the limit, as the number of experimental trials performed (trials must be performed under "identical" conditions) goes to infinity, of the number of outcomes of the event of interest $ N_A $:

$ P(A) = lim_{n \rightarrow \infty} {N_A \over N} $

Limitations:

• can never actually perform an infinite number of trials
• what does "identical" mean? e.g. if you're performing a turbulence experiment, how can you initialize everthing the exact same way?
• (Tony): what's your infinity?
  • (Sean): whatever it is, it's not finite

Probability can be seen as lack of information (e.g. fluid mechanics is deterministic, so we could describe it if we know the state perfectly - we use probability to fill in/make up for the lack of knowledge)

Quantum theory: in two-split experiment, the very state of nature is random and needs to be defined using probability

Probability of an outcome in the future: judging confidence based on prior experience... statement of confidence

Bayesian vs. fequentist

wikipedia:Noether's theorem