MeanAndVariance

Summary

This page covers mean and variance definitions for continuous random variables (with prescribed probability density function) and discrete random variables (with prescribed probability mass function).

Mean

Continuous Random Variables

If we have a continuous random variable $ X $ with a probability density function $ f(x) $, the mean and variance are given by:

$ \mu = E[X] = \int x f(x) dx $

(where the integral is over the range of x values)

Discrete Random Variable

The mean of a discrete random variable $ X $ with discrete values $ x_i, 1 \leq i \leq n $ and a probability mass function $ p_i $ is given by the expression:

$ \mu = E[X] = \sum_{i=1}^{n} p_i x_i $

Note that by definition, the probability mass function must sum to 1:

$ \sum_{i=1}^{n} p_i = 1 $

If we assume a uniform probability for each value, then the probability mass function of component i is just:

$ p_i = \frac{1}{n} $

Variance

Continuous Random Variable

The variance of a continuous random variable is given by:

$ \sigma^2 = Var(X)^2 = Var(X) = \int (x - \mu)^2 f(x) dx $

Note that this can be expanded and simplified,

$ Var(X) = \int (x^2 - 2 x \mu + \mu^2) f(x) dx = \int x^2 f(x) d - 2 \mu \int x f(x) dx + \mu^2 \int f(x) dx = \int x^2 f(x) d - 2 \mu^2 + \mu^2 = \int x^2 f(x) dx - \mu^2 $

or just

$ \sigma^2 = Var(X) = \int x^2 f(x) dx - \mu^2 $

which is equivalent to saying:

$ Var(X) = E[X^2] - E[X]^2 $

Discrete Random Variable

If we have a discrete random variable with a probability mass function,