Quantile regression

Remarks

For a real random variable $Y$, the minimisers of $b \mapsto E[\rho_\tau(Y - b)]$ are exactly the $\tau$-quantiles of $Y$ (when $E\abs{Y} < \infty$). This is why the check loss targets quantiles, just as squared loss targets the mean. The minimisation is a linear program. When $x_1, \dots, x_n$ span $\R^p$, some minimiser fits at least $p$ data points exactly. For $\tau = 1/2$, $\rho_{1/2}(u) = \abs{u}/2$, and the method is least absolute deviations regression.

Example: the intercept-only model, $p = 1$ and $x_i = 1$, with $y = (1, 2, 3, 4, 5)$ and $\tau = 1/2$. The function $b \mapsto \sum_i \abs{y_i - b}/2$ is minimised at $b = 3$, the sample median.

Non-example: minimising $\sum_i (y_i - x_i^{T}\beta)^2$ is linear regression by least squares. Its loss is quadratic, not a check function, and it estimates the conditional mean. For skewed data the conditional mean and the conditional median differ, so the two fits differ.