Estimating the relation between the covariance of a vector and a monotone function of the same vector

Question

Let $\boldsymbol{X}\in\mathbb{R}^n$ be a random variable with positive entries ($X_i\geq a>0$). I want to characterize the relation between the second moment matrix $\boldsymbol{M}$, defined as

$$ \boldsymbol{M} = \mathbb{E}\{\boldsymbol{X}\boldsymbol{X}^*\}, $$

and that we can assume to be positive definite, and the matrix

$$ \boldsymbol{N} = \mathbb{E}\{\log(\boldsymbol{X})\boldsymbol{X}^*\}, $$

where $\log(\boldsymbol{X})$ is point-wise logarithm of the entries of $\boldsymbol{X}$. In particular, I want to characterize the product of $\boldsymbol{N}$ with $\boldsymbol{M}^{-1}$:

$$ \boldsymbol{N} \boldsymbol{M}^{-1} = \color{red}{\quad ?} $$

Experimentally, by computing these statistics on my data, I find that this product appears to be equal to the sum of a multiple of the identity, plus a matrix with constant columns:

$$ \boldsymbol{N} \boldsymbol{M}^{-1} \ \stackrel{?}{\simeq}\ \alpha \boldsymbol{I}+ \boldsymbol{1}\boldsymbol{v}^* $$ Moreover, this matrix seems to be diagonally dominant.

I tried to play around with other functions other than $\log$ and it looks that this relation holds for other monotone functions as well.

I tried several approaches but I did not did not get anything out of them (Taylor expansions, copulas, etc.) though this might be due to my not particularly advanced graduate level analysis skills.

---

There are some simple transformation to the problems that do not seem to lead anywhere --- for example, $\boldsymbol{M} = \text{cov}(\boldsymbol{X},\boldsymbol{X}) + \boldsymbol{\mu}\boldsymbol{\mu}^*$, and so $\boldsymbol{M}^{-1}$ can be written using the Sherman–Morrison formula for a rank-one update: $\boldsymbol{M}^{-1} = \text{cov}(\boldsymbol{X},\boldsymbol{X})^{-1} - \boldsymbol{a}\boldsymbol{a}^*$ for some vector $\boldsymbol{a}^*$.

Eventually one arrives at estimating the product

$$ \text{cov}(\log(\boldsymbol{X}),\boldsymbol{X}) \text{cov}(\boldsymbol{X},\boldsymbol{X})^{-1} = \color{red}{\quad ?} $$

which I guess is the crux of the matter...

---

update: The Bussgang Theorem says that for stationary and Gaussian processes, we have:

$$ \text{corr}(\mathbf{X},f(\mathbf{X})) = C_f \text{corr}(\mathbf{X},\mathbf{X}) $$

where $C_f$ is a constant that depends only on $f$, which, if you ask me, it is pretty surprising.

Answer 1

I think you are probably dealing with random vectors with some symmetry. For a generic distribution the property you are trying to prove seems not to be true. Take, e.g., a random vector taking values $(1,1,1)$, $(1,2,1)$, $(1,1,2)$ with equal probabilities, and you will see that the property is violated. However, if the distribution of $X_i$ does not depend on $i$ and the distribution of the pair $(X_i,X_j)$ $(i\ne j)$does not depend on the choice of $i$ and $j$, then you might get something.