UMVUE as an optimization problem Finding UMVUE (Minimum-variance unbiased estimator) looks like an optimisation problem (minimise the variance of $\hat{\theta}$ given the constraint $\text{E}_\theta\hat{\theta}=\theta.$
I tried to apply here a standard method from optimisation but failed.
The setting: $(X_1,\dots, X_n)$ are i.i.d. random variables on the space $(\Omega,F, \mu)$ and $\rho_\theta \mu$ is the measure corresponding to the parameter $\theta.$
Let’s consider a functional
$$
L:= \int_{\Omega}\hat{\theta}^2(X_1,\dots,X_n)\rho_\theta\mu - \left(\int_{\Omega}\hat{\theta}(X_1,\dots,X_n)\rho_\theta\mu \right)^2 - 
\lambda \left( \int_{\Omega}\hat{\theta}(X_1,\dots,X_n)\rho_\theta\mu - \theta
\right )
$$
where $\lambda$ is a Lagrange multiplier corresponding to the constraint $\int_{\Omega}\hat{\theta}(X_1,\dots,X_n)\rho_\theta\mu = \theta.$ Notice that the second term is simple $\theta^2.$
Next we take a functional derivative with respect to $\hat{\theta}.$ The result (with simplified notation) is
$$
\frac{\delta L}{\delta \hat{\theta}} = 2 \int_{\Omega}\hat{\theta} \delta \hat{\theta}\rho_\theta\mu - 0 - \lambda \int_{\Omega}\delta \hat{\theta}\rho_\theta\mu =  \int_{\Omega}(2\hat{\theta}-\lambda) \delta \hat{\theta}\rho_\theta\mu.
$$
So $\frac{\delta L}{\delta \hat{\theta}}=0$ is equivalent to $2\hat{\theta}=\lambda.$ That implies (as I understand) that $\hat{\theta}$ is a constant estimator. This answer doesn’t make any sense to me.
Is it possible to modify this computation somehow in order to get something meaningful?
 A: $\newcommand\th\theta$

*

*It is not true in general that the Cramér--Rao lower bound is a solution to a meaningful optimisation problem:  "Under some regularity conditions, the Cramér-Rao lower bound is
attained iff $f_\th$ is in an exponential family" (p.14), where $(f_\th)_{\th\in I}$ is a parametric family defining the statistical model and $I$ is an open interval on the real line. In fact, an important condition is missing in this quote: if the Cramér--Rao lower bound is attained for an unbiased estimator $T(X_1,\dots,X_n)$ of $\th$, then, under certain regularity conditions, the joint density of the sample $(X_1,\dots,X_n)$ must be of the following specific form
$$f_\th(x_1,\dots,x_n)
=\exp[w(\th)T(x_1,\dots,x_n)]
c(\th)h(x_1,\dots,x_n),$$
with the same function $T$ in the exponent. So, in most cases the Cramér--Rao lower bound will not be attained.


*You cannot define an estimator $\hat\theta$ by the formula $\hat\theta(X_1,\dots,X_n):=\th$ -- because an estimator of $\th$ cannot depend on $\th$; it may depend only on the sample $X_1,\dots,X_n$.
