Kernel density estimation

Remarks

The substitution $u = (x - X_i)/h$ gives $\int_{\R} \hat f_h(x) \, dx = 1$; if moreover $K \ge 0$, then $\hat f_h$ is a probability density. Common kernels are the Gaussian kernel $K(u) = e^{-u^2/2}/\sqrt{2\pi}$ and the Epanechnikov kernel $K(u) = \frac{3}{4}(1 - u^2)$ for $\abs{u} \le 1$, $0$ otherwise. Example: with $K = \frac{1}{2} \mathbf{1}_{[-1, 1]}$, $\hat f_h(x)$ is the fraction of data in $[x - h, x + h]$ divided by $2h$. Non-example: $K = \mathbf{1}_{[-1, 1]}$ has integral $2$, so the resulting $\hat f_h$ integrates to $2$ and is not a density estimate. If $f$ is twice continuously differentiable and $K$ is a bounded symmetric density with finite variance, the mean integrated squared error is smallest for $h$ of order $n^{-1/5}$. In $\R^d$, one uses $\frac{1}{n h^d} \sum_i K((x - X_i)/h)$ with a kernel on $\R^d$.