General linear model

Builds on Linear regression Covariance matrix Expected value Multivariate normal distribution

Definition

Let $n, p, m \ge 1$ . A general linear model is a statistical model for a random $n \times m$ response matrix $Y$ of the form $Y = X B + U,$ where:

$X$ is a known real $n \times p$ matrix, the design matrix;
$B$ is an unknown real $p \times m$ matrix of coefficients;
$U$ is a random $n \times m$ error matrix whose rows are independent with expected value $0$ and a common covariance matrix $\Sigma$ .

Often the rows of $U$ are assumed to follow the multivariate normal distribution $N_m(0, \Sigma)$ .

Remarks

With $m = 1$ , the model is $y = X\beta + \varepsilon$ , multiple linear regression, and $\Sigma = \sigma^2$ is a single variance.

Example: the comparison of two group means. Let observations $1, \dots, n_1$ belong to group $1$ and the others to group $2$ , and let $X$ have two columns: the indicator of group $1$ and the indicator of group $2$ . Then $\beta = (\mu_1, \mu_2)^T$ , and the model says that $y_i$ has mean $\mu_1$ or $\mu_2$ according to its group, with a common error variance. This is the model of the two-sample $t$ -test.

Non-example: the model $E[y_i] = \exp(x_i^T \beta)$ is not a general linear model. Its mean is not a linear function of $\beta$ , so it cannot be written as the $i$ -th entry of $X B$ .

When $X$ has rank $p$ , the least squares estimate is $\hat B = (X^T X)^{-1} X^T Y$ , computed column by column.

Reading path

To understand this page from the ground up, read these 5 pages in order. The foundations come first.

Show the path as a graph

Each page uses the pages above it. The colour of the bar on a box shows how strong the evidence is, and the box also gives it as text. The list has the same pages.

no evidence level

Covariance matrix concept
Expected value concept
Linear regression concept
Multivariate normal distribution concept
General linear model definition

Tag	`GQM8GPBRYN`
URI	`m2://GQM8GPBRYN`
Kind	definition
Writing	W0 stub
MSC	62J05, 62J10, 62H12
Defines	General linear model
Uses	Linear regression, Covariance matrix, Expected value, Multivariate normal distribution

General linear model

Definition

Remarks

Reading path

Further reading

What this page needs for W1

What links here