UMVUE as an optimization problem

Question

Finding UMVUE (Minimum-variance unbiased estimator) looks like an optimisation problem (minimise the variance of $\hat{\theta}$ given the constraint $\text{E}_\theta\hat{\theta}=\theta.$

I tried to apply here a standard method from optimisation but failed. The setting: $(X_1,\dots, X_n)$ are i.i.d. random variables on the space $(\Omega,F, \mu)$ and $\rho_\theta \mu$ is the measure corresponding to the parameter $\theta.$ Let’s consider a functional $$ L:= \int_{\Omega}\hat{\theta}^2(X_1,\dots,X_n)\rho_\theta\mu - \left(\int_{\Omega}\hat{\theta}(X_1,\dots,X_n)\rho_\theta\mu \right)^2 - \lambda \left( \int_{\Omega}\hat{\theta}(X_1,\dots,X_n)\rho_\theta\mu - \theta \right ) $$ where $\lambda$ is a Lagrange multiplier corresponding to the constraint $\int_{\Omega}\hat{\theta}(X_1,\dots,X_n)\rho_\theta\mu = \theta.$ Notice that the second term is simple $\theta^2.$

Next we take a functional derivative with respect to $\hat{\theta}.$ The result (with simplified notation) is

$$ \frac{\delta L}{\delta \hat{\theta}} = 2 \int_{\Omega}\hat{\theta} \delta \hat{\theta}\rho_\theta\mu - 0 - \lambda \int_{\Omega}\delta \hat{\theta}\rho_\theta\mu = \int_{\Omega}(2\hat{\theta}-\lambda) \delta \hat{\theta}\rho_\theta\mu. $$ So $\frac{\delta L}{\delta \hat{\theta}}=0$ is equivalent to $2\hat{\theta}=\lambda.$ That implies (as I understand) that $\hat{\theta}$ is a constant estimator. This answer doesn’t make any sense to me.

Is it possible to modify this computation somehow in order to get something meaningful?

Answer 1

$\newcommand\th\theta$

1. It is not true in general that the Cramér--Rao lower bound is a solution to a meaningful optimisation problem: "Under some regularity conditions, the Cramér-Rao lower bound is attained iff $f_\th$ is in an exponential family" (p.14), where $(f_\th)_{\th\in I}$ is a parametric family defining the statistical model and $I$ is an open interval on the real line. In fact, an important condition is missing in this quote: if the Cramér--Rao lower bound is attained for an unbiased estimator $T(X_1,\dots,X_n)$ of $\th$, then, under certain regularity conditions, the joint density of the sample $(X_1,\dots,X_n)$ must be of the following specific form $$f_\th(x_1,\dots,x_n) =\exp[w(\th)T(x_1,\dots,x_n)] c(\th)h(x_1,\dots,x_n),$$ with the same function $T$ in the exponent. So, in most cases the Cramér--Rao lower bound will not be attained.

2. You cannot define an estimator $\hat\theta$ by the formula $\hat\theta(X_1,\dots,X_n):=\th$ -- because an estimator of $\th$ cannot depend on $\th$; it may depend only on the sample $X_1,\dots,X_n$.