Sample mean and sample covariance

Builds on Sample mean Sample variance Covariance matrix Statistic

Definition

Let $n \ge 2$ and $k \ge 1$ , and let $X_1, \dots, X_n$ be random vectors in $\R^k$ , written as columns. The sample mean is the statistic $\bar{X} = \frac{1}{n} \sum_{i=1}^{n} X_i \in \R^k.$ The sample covariance matrix is the $k \times k$ matrix $Q = \frac{1}{n - 1} \sum_{i=1}^{n} \left(X_i - \bar{X}\right)\left(X_i - \bar{X}\right)^T.$ Its entry $Q_{jl}$ is the sample covariance of the $j$ -th and the $l$ -th coordinates.

Remarks

Example: for the observations $(0, 0)^T$ and $(2, 2)^T$ , the mean is $(1, 1)^T$ and the deviations are $(-1, -1)^T$ and $(1, 1)^T$ . Each deviation gives the matrix with all entries $1$ , so $Q$ has all four entries equal to $2$ . For $k = 1$ , $\bar{X}$ is the sample mean and $Q$ is the sample variance. If the $X_i$ are i.i.d. with mean vector $\mu$ and finite covariance matrix $\Sigma$ , then $E[\bar{X}] = \mu$ and $E[Q] = \Sigma$ . Non-example: the matrix with divisor $n$ in place of $n - 1$ is also used, but it is not unbiased; its expectation is $\frac{n-1}{n}\Sigma$ . The matrix $Q$ is symmetric and positive semidefinite, and it is singular when $n \le k$ .

Reading path

To understand this page from the ground up, read these 70 pages in order. The foundations come first.

Construction of the reals from Cauchy sequences concept
Element definition
Integers object
Ordered field definition
Proposition concept
Random experiment concept
Rational numbers object
Real numbers object
Set definition
Ring of integers statement
Disjunction definition
Implication definition
Predicate definition
Ordered pair definition
Union definition
Sample space definition
The rationals form a field statement
Existential quantifier definition
Universal quantifier definition
Cartesian product definition
Subset definition
The rationals form an ordered field statement
Uniqueness quantifier definition
Binary relation definition
Universal set definition
Order on the reals definition
Power set definition
Complement definition
Function definition
Interval definition
Injective function definition
Preimage of a set definition
Sigma-algebra definition
Countable set definition
Event definition
Borel sigma-algebra definition
Probability axioms concept
Probability measure definition
Probability distribution definition
Probability space definition
Conditional probability definition
Random variable definition
Discrete random variable definition
Law of a random variable definition
Independence of two events definition
Random variable with a density definition
Joint probability distribution definition
Identically distributed random variables definition
Expected value definition
Mutual independence of events definition
Independence of two random variables definition
Marginal distribution definition
Variance definition
Linearity of expectation statement
Independent sigma-algebras definition
Independence means the joint law is the product of the marginals statement
Covariance definition
Independence of random variables definition
Covariance matrix definition
Variance of a sum statement
Independent and identically distributed random variables definition
Expectation of a product of independent random variables statement
Statistic definition
Independent random variables are uncorrelated statement
Estimator definition
Variance of a sum of independent random variables statement
Bias of an estimator definition
Sample mean definition
Sample variance definition
Sample mean and sample covariance definition

Tag	`9JP83HPAEX`
URI	`m2://9JP83HPAEX`
Kind	definition
Writing	W0 stub
MSC	62H12, 62F10
Defines	Sample mean and sample covariance
Uses	Sample mean, Sample variance, Covariance matrix, Statistic

Sample mean and sample covariance

Definition

Remarks

Reading path

Further reading

What this page needs for W1

What links here