I would like to understand the relation (if any) between the Cramer-Rao Lower Bound of estimation theory and the following simple definition of "reconstruction accuracy" which doesn't use any statistical framework.

Let $\theta\in\Theta\subset \mathbb{R}^n$ be an unknown parameter vector and let $x=f(\theta)\subset\mathbb{R}^m$ be a set of measurements, from which $\theta$ is to be reconstructed. Assume that the absolute error in each measurement is bounded from above by $\epsilon>0$, and otherwise nothing is known about the measurement error. Let $T_{\theta,\epsilon}=f^{-1}(x+B(0,\epsilon))$ (the full preimage, always well-defined). Obviously, $\theta\in T_{\theta,\epsilon}$. The "best possible" reconstruction accuracy is defined as $\delta_{\theta,\epsilon}=\sup_{t\in T_{\theta,\epsilon}} \|t-\theta\|$.

(Informal justification for the definition: if the measurement error is bounded by $\epsilon$, the original parameter vector $\theta$ can only be recovered up to accuracy $\delta_{\theta,\epsilon}$ because any parameter vector $\hat\theta\in T_{\theta,\epsilon}$ could have produced a measurement vector $\hat x \in B(x,\epsilon)$.)

In contrast, Cramer-Rao lower bound depends on some distribution $p(x;\theta)$ of the measurements, and gives a bound on the variance of any unbiased estimator $\hat \theta$ of $\theta$ satisfying some regularity conditions.

Apparently, the two notions are different and in general not comparable.

Still, are there cases in which the Cramer-Rao bound can be thought of as the $\delta_{\theta,\epsilon}$ defined above?


Consider the simple case of a single parameter. Assuming $f'(\theta)\neq 0$, first-order approximation of the reconstruction accuracy as defined above is $$\delta_{\theta,\epsilon}\approx \frac{\epsilon}{f'(\theta)}. \qquad (1) $$ On the other hand, assuming that the measurements are modeled as $$ x = f(\theta) + w$$ where $w$ is Gaussian with zero mean and variance $\sigma^2$, we have (see e.g. Kay's book, section 3.5) $$ CRB\{\theta\} = \frac{\sigma^2}{\biggl(\frac{\partial f}{\partial \theta}\biggr)^2} \qquad (2)$$

Comparing $(1)$ and $(2)$ seems to suggest that in this case the notions are very closely related.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.