Loss function

Remarks

For estimating a real quantity $g(\theta)$, the action set is $\mathcal{A} = \R$ and standard choices are the following. The squared error loss is $L(\theta, a) = (a - g(\theta))^2$. The absolute error loss is $L(\theta, a) = \abs{a - g(\theta)}$. In a test of $\theta \in \Theta_0$ against $\theta \notin \Theta_0$, with $\mathcal{A} = \set{0, 1}$, the $0$-$1$ loss is $1$ for a wrong decision and $0$ for a right one. Some authors allow real-valued losses that are only bounded below; this changes little in the theory.

Example: $\Theta = \mathcal{A} = \R$ and $L(\theta, a) = (a - \theta)^2$. It is a function of the pair $(\theta, a)$ with values in $[0, \infty)$, and it is $0$ exactly when $a = \theta$.

Non-example: $L(\theta, a) = a - \theta$ on $\R \times \R$. It takes negative values, so it is not a loss function in the sense above. It would also reward underestimating $\theta$ without limit.