Fisher information

Remarks

Under the usual regularity conditions, differentiation under the integral sign gives $E_\theta[s(\theta; X)] = 0$. Then $I(\theta) = \operatorname{Var}_\theta(s(\theta; X))$, the variance of the score. If moreover $\log f$ is twice differentiable, then $I(\theta) = -E_\theta\big[\partial^2_\theta \log f(X; \theta)\big]$. For $n$ independent observations with the same density, the information is $n I(\theta)$.

Example: let $X$ be normal with mean $\theta$ and known variance $\sigma^2$. Then $\log f(x; \theta) = -(x - \theta)^2 / (2\sigma^2) + c$, so $s = (X - \theta)/\sigma^2$. Hence $I(\theta) = E_\theta[(X - \theta)^2] / \sigma^4 = 1/\sigma^2$.

Non-example of the regular behaviour: the uniform densities $f(x; \theta) = 1/\theta$ on $(0, \theta)$. Here $s = -1/\theta$ wherever it is defined, so $E_\theta[s^2] = 1/\theta^2$, but $E_\theta[s] = -1/\theta \ne 0$. The support depends on $\theta$, differentiation under the integral fails, and the usual properties of $I(\theta)$ do not hold.