MeanAndVariance

Summary

This page covers mean and variance definitions in detail.

Mean

Continuous Random Variables

If we have a continuous random variable $ X $ with a probability density function $ f(x) $, the mean and variance are given by:

$ \mu = E[X] = \int x f(x) dx $

(where the integral is over the range of x values)

Discrete Random Variable

The mean of a discrete random variable $ X $ with discrete values $ x_i, 1 \leq i \leq n $ and a probability mass function $ p_i $ is given by the expression:

$ \mu = E[X] = \sum_{i=1}^{n} p_i x_i $

Note that by definition, the probability mass function must sum to 1:

$ \sum_{i=1}^{n} p_i = 1 $

If we assume a uniform probability for each value, then the probability mass function of component i is just:

$ p_i = \frac{1}{n} $

Variance

Continuous Random Variable

The variance of a continuous random variable is given by:

$ \sigma^2 = Var(X)^2 = Var(X) = \int (x - \mu)^2 f(x) dx $

Note that this can be expanded and simplified,

$ Var(X) = \int (x^2 - 2 x \mu + \mu^2) f(x) dx = \int x^2 f(x) d - 2 \mu \int x f(x) dx + \mu^2 \int f(x) dx = \int x^2 f(x) d - 2 \mu^2 + \mu^2 = \int x^2 f(x) dx - \mu^2 $

or just

$ \sigma^2 = Var(X) = \int x^2 f(x) dx - \mu^2 $

which is equivalent to saying:

$ Var(X) = E[X^2] - E[X]^2 $