MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let $\boldsymbol{X}\in\mathbb{R}^n$ be a random variable with positive entries ($X_i\geq a>0$). I want to characterize the relation between the second moment matrix $\boldsymbol{M}$, defined as

$$ \boldsymbol{M} = \mathbb{E}\{\boldsymbol{X}\boldsymbol{X}^*\}, $$

and that we can assume to be positive definite, and the matrix

$$ \boldsymbol{N} = \mathbb{E}\{\log(\boldsymbol{X})\boldsymbol{X}^*\}, $$

where $\log(\boldsymbol{X})$ is point-wise logarithm of the entries of $\boldsymbol{X}$. In particular, I want to characterize the product of $\boldsymbol{N}$ with $\boldsymbol{M}^{-1}$:

$$ \boldsymbol{N} \boldsymbol{M}^{-1} = \color{red}{\quad ?} $$

Experimentally, by computing these statistics on my data, I find that this product appears to be equal to the sum of a multiple of the identity, plus a matrix with constant columns:

$$ \boldsymbol{N} \boldsymbol{M}^{-1} \ \stackrel{?}{\simeq}\ \alpha \boldsymbol{I}+ \boldsymbol{1}\boldsymbol{v}^* $$ Moreover, this matrix seems to be diagonally dominant.

I tried to play around with other functions other than $\log$ and it looks that this relation holds for other monotone functions as well.

I tried several approaches but I did not did not get anything out of them (Taylor expansions, copulas, etc.) though this might be due to my not particularly advanced graduate level analysis skills.

There are some simple transformation to the problems that do not seem to lead anywhere --- for example, $\boldsymbol{M} = \text{cov}(\boldsymbol{X},\boldsymbol{X}) + \boldsymbol{\mu}\boldsymbol{\mu}^*$, and so $\boldsymbol{M}^{-1}$ can be written using the Sherman–Morrison formula for a rank-one update: $\boldsymbol{M}^{-1} = \text{cov}(\boldsymbol{X},\boldsymbol{X})^{-1} - \boldsymbol{a}\boldsymbol{a}^*$ for some vector $\boldsymbol{a}^*$.

Eventually one arrives at estimating the product

$$ \text{cov}(\log(\boldsymbol{X}),\boldsymbol{X}) \text{cov}(\boldsymbol{X},\boldsymbol{X})^{-1} = \color{red}{\quad ?} $$

which I guess is the crux of the matter...

update: The Bussgang Theorem says that for stationary and Gaussian processes, we have:

$$ \text{corr}(\mathbf{X},f(\mathbf{X})) = C_f \text{corr}(\mathbf{X},\mathbf{X}) $$

where $C_f$ is a constant that depends only on $f$, which, if you ask me, it is pretty surprising.

share|cite|improve this question
up vote 0 down vote accepted

I think you are probably dealing with random vectors with some symmetry. For a generic distribution the property you are trying to prove seems not to be true. Take, e.g., a random vector taking values $(1,1,1)$, $(1,2,1)$, $(1,1,2)$ with equal probabilities, and you will see that the property is violated. However, if the distribution of $X_i$ does not depend on $i$ and the distribution of the pair $(X_i,X_j)$ $(i\ne j)$does not depend on the choice of $i$ and $j$, then you might get something.

share|cite|improve this answer
You are right. I just realized that the data I have has a notable symmetry. Each element of the vector has the same distribution: $p(X_i = x) = p(X_j = x)$. However, they are not independent variables. The joint distribution of $(X_i, X_j)$ does depend on the choice of $i,j$, however. (it depends on the distance $d(s_i, s_j)$ between two points $s_i$, $s_j$ in a certain metric space. Think of $s_i$ as a spatial position of a sensor that generates the data $X_i$.) Do you have any pointer to similar problems that I can investigate? – Andrea Censi Jul 16 '13 at 13:06
This is similar then to stationary random processes or even isotropic random fields. – Yuri Bakhtin Jul 16 '13 at 18:27
D'oh! I should have spotted stationarity as the relevant property. Time to delve into some class 1950s mathematical statistics... on a first google expedition, it looks like very strong results can be proved similar to my hunch. E.g. On a cross-correlation property for stationary random processes – Andrea Censi Jul 16 '13 at 23:02

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.