refrence for simple statistics question: estimation of variance of $n$, for known $r=s+n$, for $s= \pm1$, $n=N(0,\sigma)$

Question

Assume someone generate random number "s"(signal) which is $+1$ or $-1$ (uniformly distributed), and generate random number $n$ (noise) $N(0,\sigma)$ (i.e. normal distributed with mean zero, variance $\sigma^2$). Assume "$s$" and "$n$" are independent.

Then "he" sums: $r = s+n$ and gives number "$r$" to you.

Assume you have a sample of length "$n$" of numbers $r_1,\ldots, r_n$.

Question you want to estimate sigma - how to do it?

More precisely I need simple estimation, i.e. such that complexity of the algorithm will be more or less the same as for simple solutions (1 , 2 ) described below, but I want more higher accuracy than the naive solutions below.

If there some nice reference on this it would be helpful... "Simple" solutions are given below.

(Clarifications: you do not know "s_i" not "n_i", but you know that distribution of "s" and knows what "n" is N(0,sigma) , but you do not know sigma) ?

---

Motivation. Actually yours mobile phones solves this (or more complicated) problem every millisecond (or so). That is why complexity is an issue. "s" is sent bit, during propagation from base station to yours mobile phone it is spoiled. The simplest mathematical model of such propagation is to add random normal noise: so "r" - is received signal. Mobile phone wants to determine what have been sent +1 or -1. Now some profound algorithms first need estimation of noise level "n". After that they start determining "s".

---

Solution 1.

Denote by D(ksi) - variance of randome variable ksi.

So we know that D(r) = D(s) + D(n). D(s) is know to us D(s) =1. So we can estimate:

D(n) = D(r) -1 .

So use standard estimate of D(r) and just subtract "1" - and that is all.

Very simple ! But this estimate is rather bad - it can be negative for example and its variance is very far from ideal.

---

Solution 2.

Take numbers r_i and form numbers se_i = sign(r_i) (these are estimate of "s"). Calculate ne_i = r_i - se_i (these are estimates of noise).

Calculate D(ne_i) - take this as an answer...

This solution is actually perfect is "sigma" is small, but if "sigma" is large it is biased. I.e. mean of this estimate is not true sigma.

---

Solution 3.

I tried MLD solution. But to my surprise it is also biased for big noise. It leads to solution of transcendental equation. So the problems of complexity arises...

Answer 1

You could try a moment-matching approach. In particular, define $\hat{s}$ to be the value of $s$ such that

$$\mbox{Pr}_{s}(-1 < r < 1) = \frac{\sum_i I(-1 < r_i < 1)}{N}$$

where $N$ is the number of observations. You can calculate the left hand side explicitly as $$\mbox{Pr}_s(-1 < r <1 \mid S = 1)\mbox{Pr}(S = 1) + \mbox{Pr}_s(-1< r <1 \mid S = -1)\mbox{Pr}(S = -1)$$

which simplifies (I think) to

$$\Phi(2/s) - 1/2,$$

where $\Phi(\cdot)$ denotes the cumulative distribution function for a standard normal random variable. Inverting this gets

$$\hat{s} = \frac{2}{\Phi^{-1}\left(\frac{\sum_i I(-1 < r_i < 1)}{N} + 1/2\right)}.$$

I'm not sure if this differs much from your Solution 3 in terms of complexity, but one evaluation of the inverse CDF of a Gaussian isn't so bad.

Answer 2

You can do it by essentially iterating (2). First you get your sigma estimate using (2). This is not so great as you've said. Now that you have a (bad) sigma estimate, you can use this to get a first-approximation posterior distribution over the -1, +1 values. So instead of assigning them hard values according to their sign, you can use the gaussian pdf to probabilistically assign them -1 or 1 based on the bad sigma estimate. Then you can re-estimate the posterior expected sigma based on these fuzzy -1, +1 re-estimates. You can iterate between re-estimating sigma and re-estimating the soft -1, +1 distribution.

This is an application of the expectation-maximization principle, so this algorithm will converge to a local maximum likelihood estimate of sigma. Furthermore the likelihood landscape of sigma seems plain enough to not have misleading local optima, so this should also converge to the global maximum likelihood (and therefore statistically consistent) estimate of sigma.