Smoothing spline

Remarks

The minimizer exists and is unique, and it is a natural cubic spline with knots $x_1, \dots, x_n$: a cubic on each $[x_i, x_{i+1}]$, of class $C^2$, and linear on $[a, x_1]$ and $[x_n, b]$. Uniqueness uses $n \ge 2$: then a function with $f'' = 0$ that vanishes at all nodes is zero. Example: if the data lie on a line, $y_i = \alpha + \beta x_i$, then $f(x) = \alpha + \beta x$ gives $J_\lambda(f) = 0$, so it is the smoothing spline for every $\lambda$. Non-example: take $n \ge 3$ data not on a line, and let $g$ be the natural cubic spline through them. Its second derivative penalty is positive, and moving $g$ slightly toward a line lowers $J_\lambda$, so $g$ is not the minimizer. As $\lambda \to 0$ the smoothing spline tends to the natural interpolating cubic spline, and as $\lambda \to \infty$ it tends to the least squares regression line. In practice $\lambda$ is often chosen by generalized cross-validation.