1
$\begingroup$

I am confused about the definition of the Fisher information and the case when probability is 0. Consider discrete set $\epsilon$ of possible measurement outcomes. Fisher information is defined as: $$F(\theta) = \sum\limits_{\epsilon} \frac{1}{p(\epsilon|\theta)}\left( \frac{\partial p(\epsilon|\theta)}{\partial \theta}\right)^2.$$

What happens when in the sum one of the probabilities $p(\epsilon_0|\theta)=0$. I can imagine 2 situations:

  1. $p(\epsilon_0|\theta) = 0$ and $\frac{\partial p(\epsilon_0|\theta)}{\partial \theta} \neq 0$. Then $F(\theta) = \infty$.
  2. $p(\epsilon_0|\theta) = 0$ and $\frac{\partial p(\epsilon_0|\theta)}{\partial \theta} = 0$. Then we have $0/0$ which might be determined from L'Hospital rule and give finite value. Then $$\frac{1}{p(\epsilon_0|\theta)}\left( \frac{\partial p(\epsilon_0|\theta)}{\partial \theta}\right)^2 = 2 \frac{\partial^2 p(\epsilon_0|\theta)}{\partial \theta^2}.$$

Does it really make sense to take into account measurement outcome $\epsilon_0$ for which $p(\epsilon_0|\theta) = 0$ based on the fact that it does not contribute to the variance of the estimator?

$\endgroup$
1
$\begingroup$

An infinite Fisher information $F(\theta)$ means that the lower bound $1/F(\theta)$ in the error of an unbiased estimator of $\theta$ is zero. A simple example is the Bernoulli trial, success or failure with probabilities $\theta$ and $1-\theta$. The Fisher information is $$F(\theta)= \frac{1}{\theta}\frac{1}{1-\theta}$$ So $F(\theta)$ diverges for $\theta=0$ or $1$, which makes sense because then there are no fluctuations in the outcome so there is no limit to the precision with which you can predict it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.