Generalized linear model

Remarks

In the random component, $\mu_i = b'(\theta_i)$ and $\operatorname{Var}(y_i) = a(\phi) \, b''(\theta_i)$. The link $g$ with $g(b'(\theta)) = \theta$ is the canonical link; with it, $\theta_i = x_i^T \beta$.

Example: the normal distribution $N(\mu, \sigma^2)$ has the form of clause 1 with $\theta = \mu$, $b(\theta) = \theta^2/2$ and $a(\phi) = \phi = \sigma^2$. With the identity link, $\mu_i = x_i^T \beta$, and the model is normal linear regression. Example: for Bernoulli responses with the logit link $g(\mu) = \log(\mu/(1 - \mu))$ one gets logistic regression; for Poisson responses with the log link one gets Poisson regression.

Non-example: the model $y_i = x_i^T \beta + \varepsilon_i$ with independent standard Cauchy errors $\varepsilon_i$ is not a generalized linear model. The Cauchy location family is not of the form of clause 1, and $y_i$ has no expected value, so clause 3 cannot hold either.

The coefficients are estimated by maximum likelihood; with the canonical link the likelihood equations are $\sum_i (y_i - \mu_i) x_i = 0$.