Mercer's theorem

Remarks

Positivity of the kernel gives $\ip{T_K \varphi}{\varphi} \ge 0$ for all $\varphi$, so all eigenvalues are nonnegative. Positivity of the remainder kernel $K(x, y) - \sum_{n \le m} \mu_n e_n(x) e_n(y)$ gives $\sum_{n \le m} \mu_n e_n(x)^2 \le K(x, x)$ for every $m$. So the diagonal series converges pointwise, and one shows its sum is $K(x, x)$, a continuous function. Dini's theorem then makes the convergence on the diagonal uniform, and the Cauchy–Schwarz inequality transfers this to the whole square. Part 3 is part 2 on the diagonal, integrated.

Example: $K(x, y) = \min(x, y)$ on $[0, 1]$ is the covariance of Brownian motion, hence positive semidefinite. Its expansion is $\min(x, y) = \sum_{n \ge 1} \frac{2 \sin\left((n - \tfrac12)\pi x\right) \sin\left((n - \tfrac12)\pi y\right)}{(n - \tfrac12)^2 \pi^2}$, uniformly on the square. On the diagonal, part 3 gives $\sum_{n \ge 1} \frac{1}{(n - \frac12)^2 \pi^2} = \int_0^1 x \, dx = \frac12$.

Without positivity, the conclusions can fail: for a general continuous symmetric kernel the expansion is only guaranteed in $L^2$, and $\sum \abs{\mu_n}$ may diverge.

Going deeper

For a student

The theorem upgrades the spectral theorem for compact self-adjoint operators, which gives only $L^2$ convergence, to uniform convergence of the kernel itself. It is the basis of the Karhunen–Loève expansion of a stochastic process with continuous covariance. In machine learning it explains why a positive kernel is an inner product of feature vectors $(\sqrt{\mu_n} e_n(x))_n$, which leads to reproducing kernel Hilbert spaces.