# Estimating the relation between the covariance of a vector and a monotone function of the same vector

Let $\boldsymbol{X}\in\mathbb{R}^n$ be a random variable with positive entries ($X_i\geq a>0$). I want to characterize the relation between the second moment matrix $\boldsymbol{M}$, defined as

$$\boldsymbol{M} = \mathbb{E}\{\boldsymbol{X}\boldsymbol{X}^*\},$$

and that we can assume to be positive definite, and the matrix

$$\boldsymbol{N} = \mathbb{E}\{\log(\boldsymbol{X})\boldsymbol{X}^*\},$$

where $\log(\boldsymbol{X})$ is point-wise logarithm of the entries of $\boldsymbol{X}$. In particular, I want to characterize the product of $\boldsymbol{N}$ with $\boldsymbol{M}^{-1}$:

$$\boldsymbol{N} \boldsymbol{M}^{-1} = \color{red}{\quad ?}$$

Experimentally, by computing these statistics on my data, I find that this product appears to be equal to the sum of a multiple of the identity, plus a matrix with constant columns:

$$\boldsymbol{N} \boldsymbol{M}^{-1} \ \stackrel{?}{\simeq}\ \alpha \boldsymbol{I}+ \boldsymbol{1}\boldsymbol{v}^*$$ Moreover, this matrix seems to be diagonally dominant.

I tried to play around with other functions other than $\log$ and it looks that this relation holds for other monotone functions as well.

I tried several approaches but I did not did not get anything out of them (Taylor expansions, copulas, etc.) though this might be due to my not particularly advanced graduate level analysis skills.

There are some simple transformation to the problems that do not seem to lead anywhere --- for example, $\boldsymbol{M} = \text{cov}(\boldsymbol{X},\boldsymbol{X}) + \boldsymbol{\mu}\boldsymbol{\mu}^*$, and so $\boldsymbol{M}^{-1}$ can be written using the Sherman–Morrison formula for a rank-one update: $\boldsymbol{M}^{-1} = \text{cov}(\boldsymbol{X},\boldsymbol{X})^{-1} - \boldsymbol{a}\boldsymbol{a}^*$ for some vector $\boldsymbol{a}^*$.

Eventually one arrives at estimating the product

$$\text{cov}(\log(\boldsymbol{X}),\boldsymbol{X}) \text{cov}(\boldsymbol{X},\boldsymbol{X})^{-1} = \color{red}{\quad ?}$$

which I guess is the crux of the matter...

update: The Bussgang Theorem says that for stationary and Gaussian processes, we have:

$$\text{corr}(\mathbf{X},f(\mathbf{X})) = C_f \text{corr}(\mathbf{X},\mathbf{X})$$

where $C_f$ is a constant that depends only on $f$, which, if you ask me, it is pretty surprising.

-

I think you are probably dealing with random vectors with some symmetry. For a generic distribution the property you are trying to prove seems not to be true. Take, e.g., a random vector taking values $(1,1,1)$, $(1,2,1)$, $(1,1,2)$ with equal probabilities, and you will see that the property is violated. However, if the distribution of $X_i$ does not depend on $i$ and the distribution of the pair $(X_i,X_j)$ $(i\ne j)$does not depend on the choice of $i$ and $j$, then you might get something.
You are right. I just realized that the data I have has a notable symmetry. Each element of the vector has the same distribution: $p(X_i = x) = p(X_j = x)$. However, they are not independent variables. The joint distribution of $(X_i, X_j)$ does depend on the choice of $i,j$, however. (it depends on the distance $d(s_i, s_j)$ between two points $s_i$, $s_j$ in a certain metric space. Think of $s_i$ as a spatial position of a sensor that generates the data $X_i$.) Do you have any pointer to similar problems that I can investigate? – Andrea Censi Jul 16 '13 at 13:06