1
$\begingroup$

I asked the following question in a forum more suitable for statistics, but I didn't get any answer; I hope, someone could shed light on my question:

I have three random variables, $X_1$, $X_2$, and $X_3$, which they are distributed normally. If we consider an estimator which reads as: $$\hat{\theta} = a_1 X_1 + a_2 X_2 +a_3X_3,$$ with the constraint: $a_1 + a_2 + a_3 = 1$, we know that for $a_i \propto 1/\sigma_i^2$, where $\sigma_i$'s are the standard deviations of $X_i$'s, the estimator has the smallest variance (for proof, see Theorem 3.2., here).

Now, my question is: What is the significance of choosing the coefficients in the above sum as Jeffreys' priors, that is, $a_i \propto 1 / \sigma_i$?

Does $a_i \propto 1 / \sigma_i$ result, for example, in the largest acceptable variance of the estimator? Perhaps, one can say, in the case of complete ignorance, i.e., uninformative priors, the variance of $\hat{\theta}$ should be the largest acceptable variance. Is this argument valid?

$\endgroup$

1 Answer 1

1
$\begingroup$

The coefficients $a_i$ of the random variables $X_i$ are not any prior probabilities at all -- because prior probabilities are coefficients, not of random variables, but of probability distributions.

The choice $a_i\propto 1/\sigma_i$ in your setting equalizes the variances of the random variables $a_iX_i$, and that is all it does.


Even though priors have actually nothing to do with your question, for criticisms of Jeffreys' "noninformative" priors one may want to see e.g. Section 4.1.

$\endgroup$
3
  • $\begingroup$ @Dave : You can interpret the $a_i$'s as probabilities, but they are not any prior probabilities -- because, to reiterate, prior probabilities are coefficients, not of random variables (r.v.'s), but of probability distributions. A r.v. is a measurable map, whereas a probability distribution is a probability measure defined on a $\sigma$-algebra. You can also think of prior probabilities as coefficients of pdf's. But again, a pdf is not a r.v. $\endgroup$ Sep 27, 2021 at 17:54
  • $\begingroup$ @Dave : If the $X_i$'s are independent, then $S:=\sum_i a_iX_i$ is also normal. The Fisher information of a normal location family is the reciprocal of its variance; other kinds of information should also be decreasing as the variance increases. So, minimizing information should usually mean minimizing the variance, and the variance of $S$ is minimized if $a_i\propto1/\sigma_i^2$, as you noted. $\endgroup$ Sep 27, 2021 at 19:45
  • $\begingroup$ @Dave : Oops! I should have said "maximizing information". We don't want to minimize information, but if we did, we would maximize the variance of $S$, which occurs when an $a_i$ corresponding to a largest $\sigma_i$ is $1$, and all the other $a_i$'s are $0$. $\endgroup$ Sep 27, 2021 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.