Linear discriminant analysis

Remarks

If the class $k$ has the distribution $N_p(\mu_k, \Sigma)$ and prior probability $\pi_k$, this rule is the Bayes rule. Indeed, $\delta_k(x)$ differs from $\log(\pi_k f_k(x))$ by a term that does not depend on $k$, where $f_k$ is the density of class $k$. Each $\delta_k$ is affine in $x$, so the boundary between two classes is a hyperplane. For two classes, $x$ goes to class $1$ when $$(\mu_1 - \mu_2)^T \Sigma^{-1} x - \tfrac{1}{2} (\mu_1 - \mu_2)^T \Sigma^{-1} (\mu_1 + \mu_2) > \log(\pi_2 / \pi_1).$$ Example: with $p = 1$, $\mu_1 = 1$, $\mu_2 = -1$, $\Sigma = 1$ and $\pi_1 = \pi_2$, the condition is $2x > 0$, so $x$ goes to class $1$ when $x > 0$. Non-example: if the normal classes have different covariance matrices $\Sigma_k$, the Bayes rule has quadratic boundaries. This rule is quadratic discriminant analysis, not a linear rule.