I have a linear model $$Y=HX+N,$$ where $H$ is a matrix and $X$ are drawn from $p_X(X)$, and $N$ is Gaussian noise variates.

Now, if $X$ is multivariate Gaussian, then a linear estimator $\hat{X}=A+BX$ solves the problem $$\mathcal{E}(|\hat{X}-X|^2).$$ The matrix $B$ is the standard Wiener filter, and $A$ is only activated if $\mathcal{E}(X)\neq 0$.

Now, for non-Gaussian inputs, a linear estimator is not optimal.

However, my intuition tells me that the solution could be expressed through a sum of cumulants. My reasoning is that since only the two first cumulants are non-zero for Gaussian, this is also the reason why a linear estimator $A+BX$ is optimal for Gaussian inputs $X$.

Is there anyone that have been thinking about this problem? (or is willing to do so?)

  • $\begingroup$ MMSE estimator can ALWAYS be expressed as conditional mean E[X|Y]. Have you tried to compare yours conjecture with this ? E.g. for X_i =+-1 (BPSK modulation) ? $\endgroup$ Sep 6 '12 at 19:39
  • $\begingroup$ not really... Of course, the solution is clearly $E[X|y]$, and my idea is that it would be possible to express this as a sum of cumulants. But so far, I don't have any good idea on how to attack the problem. I did some searches, but found nothing. $\endgroup$ Sep 6 '12 at 19:47
  • 1
    $\begingroup$ Cumulants are statistical deterministic measures, so "sum of cumulants" can't express conditional expectation which is a "random variable". If you meant that $E(X\mid Y)$ can be represented as, for example, a power series, then I believe that Alexander's example contradict your conjecture. $\endgroup$
    – Josh
    Sep 7 '12 at 14:23
  • $\begingroup$ ok, let me rephrase my question: for $x$ distributed as a multivariate Gaussian vector, the expectation $E(X|Y)$ is linear in Y (plus a constant). I Believe that this could be seen from the fact that only the two first cumulants of a Gaussian distribution are non-zero. What I am after here is an "update" formula for the linear MMSE of non-Gaussian $X$, expressed through cumulants. $\endgroup$ Sep 17 '12 at 21:33

I have given a similar issue some thought and it seems that the answer to your question is on the negative in general but positive when the input has constant norm. To gain intution consider the simple case of a scalar Gaussian channel with unit noise variance and discrete input with alphabet $\mathcal X$ and probability mass function $p(x)$ for every $x \in \mathcal X$. We then have

$$E[X|Y=y]=\frac{\displaystyle\sum_{x \in \mathcal X}xp(x)e^{-\frac{1}{2}(y-x)^2}}{\displaystyle\sum_{x \in \mathcal X}p(x)e^{-\frac{1}{2}(y-x)^2}}=\frac{d}{dy}\log\left( \sum_{x \in \mathcal X}p(x)e^{-\frac{1}{2}x^2 + xy}\right).$$ Now, notice that $\log(\sum_{x \in \mathcal X}p(x)e^{xy})$ is the comulant generating function of $X$ at point $y$. Therefore, if $X^2$ is deterministic then the conditional expectation can indeed be expressed as a power series in $y$ with coefficients that are the comulants of $X$. Otherwise, the conditional expectation can still be expanded to a power series in $y$ for fun and profit, but the coefficients are not likely to depend on the cumulants in any simple way, at least as far as I see.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.