1
$\begingroup$

This is a follow-up to my previous question.

Assume that $X\sim \mathcal N(\mu_1,\sigma_1^2)$ and $Y\sim \mathcal N(\mu_2,\sigma_2^2)$.

I want to estimate $\frac{\mu_1+\mu_2}{2}$ after observing $X,Y$.

In my setting, $\sigma_1,\sigma_2$ are known and we want to estimate the average of the means with the goal of minimizing the expected squared error.


Intuitively, if $\sigma_1<\sigma_2$, we should have an estimate that's closer to $X$ than to $Y$. How can we formalize this?

$\endgroup$
9
  • 1
    $\begingroup$ Without restrictions on $\mu_1,\mu_2$, I think you will not get the desired result. $\endgroup$ Commented Oct 21, 2021 at 13:34
  • $\begingroup$ What is the setting and where does your intuition come from? I suspect that your interest here is not purely mathematical, and you’d probably get more insightful answers by providing more context. $\endgroup$
    – user44143
    Commented Oct 21, 2021 at 13:39
  • $\begingroup$ Thanks, @IosifPinelis. Can you please explain why? Let's even say that $\sigma_1=0$; wouldn't it make more sense to have an estimate closer to $X$? $\endgroup$
    – R B
    Commented Oct 21, 2021 at 13:47
  • $\begingroup$ @Matt, The problem comes from trying to estimate the average of measurements taken by heterogeneous devices that have different bandwidths for transmitting their signal. This means that the devices have different quantization errors, which we can estimate as we know the number of bits they transmitted. To simplify the model, we can assume that the quantization noise is Gaussian, although this is not accurate (and depends on the quantization method). $\endgroup$
    – R B
    Commented Oct 21, 2021 at 13:49
  • $\begingroup$ Even in that context the MLE estimate would be the average of X and Y, and I don’t see any other principled estimation method that would give anything different. $\endgroup$
    – user44143
    Commented Oct 21, 2021 at 14:13

1 Answer 1

1
$\begingroup$

$\newcommand\si\sigma$Clearly, the best estimator of $\mu_1$ is $X$, no matter what $\si_1$ and $\si_2$ are. Similarly, the best estimator of $\mu_2$ is $Y$, no matter what $\si_1$ and $\si_2$ are. So, one may argue, a good estimator of $(\mu_1+\mu_2)/2$ is the substitution estimator $(X+Y)/2$.

As was shown in the previous answer, $(X+Y)/2$ is indeed the maximum likelihood estimator of $(\mu_1+\mu_2)/2$. It is also easy to see that $(X+Y)/2$ is the only unbiased linear estimator (of the form $aX+bY$) of $(\mu_1+\mu_2)/2$, as well as the minimax linear estimator of $(\mu_1+\mu_2)/2$ (with respect to the quadratic loss function).

On the other hand, one may argue that, if $\si_1<\si_2$, then the uncertainly about $\mu_1$ is less than that about $\mu_2$, and so $X$ has to be given a greater weight than $Y$. However, if there are no restrictions on $\mu_1,\mu_2$, then $|\mu_1|$ and $|\mu_2|$ can be much greater than both $\si_1$ and $\si_2$, so that $\si_1$ and $\si_2$ will matter little, if at all.

Thus, as it was said in a comment, without restrictions on $\mu_1,\mu_2$, you will hardly get the desired result.

One way to impose (soft/fuzzy) restrictions on $\mu_1,\mu_2$ is to suppose that $(\mu_1,\mu_2)$ has the prior bivariate normal distribution with means $0,0$, variances $b^2,b^2$, and correlation $0$. Then the Bayes estimator of $(\mu_1,\mu_2)$ (assuming the quadratic loss function) is $$(\hat\mu_1,\hat\mu_2):=\Big(\frac X{1+\sigma_1^2/b^2},\frac Y{1+\sigma_2^2/b^2}\Big).$$ Then the corresponding estimator of $(\mu_1+\mu_2)/2$ is $$\frac{\hat\mu_1+\hat\mu_2}2=\frac12\,\Big(\frac X{1+\sigma_1^2/b^2}+\frac Y{1+\sigma_2^2/b^2}\Big).$$ Here, as desired, the weight/coefficient of $X$ is greater than that of $Y$ if $\sigma_1<\sigma_2$.

If you now let $b\to\infty$, thus making the prior knowledge (i.e., the restrictions on $\mu_1,\mu_2$) insignificant, then you get back the nice old estimator $(X+Y)/2$ from the previous answer.

$\endgroup$
3
  • $\begingroup$ That's helpful, thank you. $\endgroup$
    – R B
    Commented Oct 21, 2021 at 16:50
  • $\begingroup$ Another question: would the following model be reasonable: Say that we know that $\mu_1,\mu_2\in[0,1]$ and would want to minimize the expected squared error for the worst-case $\mu1,\mu_2$. Would this allow us to do something interesting? This seems closer to the problem we have than assuming a prior distribution on $\mu1,\mu2$. $\endgroup$
    – R B
    Commented Oct 22, 2021 at 9:22
  • $\begingroup$ @RB : Yes, you may want to consider such rigid, non-soft restrictions as well. $\endgroup$ Commented Oct 22, 2021 at 10:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .