Math 2.0

Statistics Definition

Generalized least squares

Builds on General linear modelOrdinary least squaresCovariance matrixPositive definite matrixGauss–Markov theorem

Definition

Let XX be a fixed real n×pn \times p matrix of rank pp, and let y=Xβ+εy = X\beta + \varepsilon with β∈ℝp\beta \in \R^p unknown. Assume E[ε]=0E[\varepsilon] = 0 and Cov⁡(ε)=σ2Ω\operatorname{Cov}(\varepsilon) = \sigma^2 \Omega, where σ2>0\sigma^2 > 0 and Ω\Omega is a known symmetric positive definite n×nn \times n matrix. The generalized least squares estimator of β\beta is the minimiser of (y−Xb)TΩ−1(y−Xb)(y - X b)^T \Omega^{-1} (y - X b) over b∈ℝpb \in \R^p, which is β^GLS=(XTΩ−1X)−1XTΩ−1y.\hat\beta_{\mathrm{GLS}} = (X^T \Omega^{-1} X)^{-1} X^T \Omega^{-1} y.

Remarks

Equivalent form: let PP be an invertible matrix with PTP=Ω−1P^T P = \Omega^{-1}, for example P=Ω−1/2P = \Omega^{-1/2}. Then Py=PXβ+PεP y = P X \beta + P \varepsilon with Cov⁡(Pε)=σ2In\operatorname{Cov}(P\varepsilon) = \sigma^2 I_n. The estimator β^GLS\hat\beta_{\mathrm{GLS}} is the ordinary least squares estimator of this transformed model. By the Gauss–Markov theorem applied to the transformed model, it is the best linear unbiased estimator; this is Aitken's theorem. Its covariance matrix is σ2(XTΩ−1X)−1\sigma^2 (X^T \Omega^{-1} X)^{-1}.

Example: if Ω=In\Omega = I_n, the estimator reduces to ordinary least squares. If Ω\Omega is diagonal, it is the weighted least squares estimator with weights 1/Ωii1/\Omega_{ii}.

Non-example: if Ω\Omega is only positive semidefinite and singular, then Ω−1\Omega^{-1} does not exist and the formula is not defined.

When Ω\Omega is unknown and is replaced by an estimate, the method is called feasible generalized least squares.

Reading path

To understand this page from the ground up, read these 10 pages in order. The foundations come first.

Show the path as a graph
Bias of an estimator, concept, no evidence levelBias of an estimatorconceptCovariance matrix, concept, no evidence levelCovariance matrixconceptGeneral linear model, concept, no evidence levelGeneral linear modelconceptHeteroscedasticity, concept, no evidence levelHeteroscedasticityconceptOrdinary least squares, concept, no evidence levelOrdinary least squaresconceptPositive definite matrix, concept, no evidence levelPositive definite mat…conceptPositive semidefinite matrix, concept, no evidence levelPositive semidefinite…conceptVariance, concept, no evidence levelVarianceconceptGauss–Markov theorem, statement, no evidence levelGauss–Markov theoremstatementGeneralized least squares, definition, no evidence levelGeneralized least squ…definition
Each page uses the pages above it. The colour of the bar on a box shows how strong the evidence is, and the box also gives it as text. The list has the same pages.
  • no evidence level
  1. Bias of an estimator concept
  2. Covariance matrix concept
  3. General linear model concept
  4. Heteroscedasticity concept
  5. Ordinary least squares concept
  6. Positive definite matrix concept
  7. Positive semidefinite matrix concept
  8. Variance concept
  9. Gauss–Markov theorem statement
  10. Generalized least squares definition

Further reading

  1. Web https://en.wikipedia.org/wiki/Generalized_least_squares
About this page
TagXK7W504YEW
URIm2://XK7W504YEW
Kinddefinition
WritingW0 stub
MSC62J05
DefinesGeneralized least squares
UsesGeneral linear model, Ordinary least squares, Covariance matrix, Positive definite matrix, Gauss–Markov theorem

Markdown · JSON · Edit on GitHub

What this page needs for W1

  • an inventory row: the row of the inventory that the node accounts for (0 of 1)
  • a source with a locator (0 of 1)

The writing levels say how complete the text of a page is. They do not measure its truth.

What links here

No page links to this page yet.