MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If $\mathbf{x}\sim\mathbf{N}(\mathbf{0},\mathbf{I})$, and assume that $A$ is a symmetric positive definite matrix, how can I calculate the following two expectations where there is a logarithm in it? $$ \mathrm{E}_\mathbf{x}\left[\log(\mathbf{x}^\top A\mathbf{x})\right] $$ and

$$ \mathrm{E}_\mathbf{x}\left[(\mathbf{x}^\top A\mathbf{x})\log(\mathbf{x}^\top A\mathbf{x})\right] $$

Using the decomposition $A=U\Lambda U^\top$ and $\mathbf{y}=U^\top\mathbf{x}$, the first expectation can be reduced to $$ \mathrm{E}_\mathbf{y}\left[\log(\mathbf{y}^\top \Lambda\mathbf{y})\right] $$

$$ =\mathrm{E}_\mathbf{y}\left[\log\left(\sum_i{\lambda_i\mathbf{y}_i^2}\right)\right] $$ where $y$ has the same distribution as $x$ because $U$ is a orthogonal matrix.

However, since the random variables $y_i$ are in the logarithm function, I cannot decompose the expectation further and such a expectation seems to be difficult to calculate.

Could you please help me with this problem or give some suggestions about it? Thank you very much!

share|cite|improve this question
What's the distribution $\bf N(0,I)$? – Noam D. Elkies Nov 21 '11 at 17:43
@Noam: He means multivariate normal with variance matrix $\mathbf{I}.$ – Igor Rivin Nov 21 '11 at 17:49
This has nice forms for 1D, but in general maybe numerical integration might be the way to go. – Suvrit Nov 21 '11 at 19:50
@Rivin: Yes, you are right. I mean multivariate normal with identity variance. – ppyang Nov 22 '11 at 1:55
up vote 2 down vote accepted

For the case $\lambda_1=\cdots=\lambda_k=1$, $\sum_i \lambda_i y_i^2$ has a chi-squared distribution with $k$ degrees of freedom. Plugging in the chi-squared density and cranking up Maple, we get $$E\Bigl(\log\bigl(\sum_{i=1}^k y_i^2\bigl)\Bigl) = \ln 2+\Psi(k/2),$$ where $\Psi(x)$ is the digamma function (the derivative of the gamma function $\Gamma(x)$). The first six values are $-\gamma-\ln 2$, $-\gamma+\ln 2$, $-\gamma+2-\ln 2$, $-\gamma+1+\ln 2$, $-\gamma+\frac83-\ln 2$, $-\gamma+\frac32+\ln 2$.

If the $\lambda_i$ are not all equal, $\sum_i \lambda_i y_i^2$ is called a weighted chi-squared distribution (and probably other names too). I don't know if there is an expression for its density that is useful here.

share|cite|improve this answer
The solution seems to be more complex than what I had expected. Thank you for help! – ppyang Nov 22 '11 at 2:19

Well, with Maple's help I can do the cases $n=1$ and $n=2$:

$E[\log(\lambda_1 Y_1^2)] = \log(\lambda_1/2) -\gamma$

$E[\log(\lambda_1 Y_1^2 + \lambda_2 Y_2^2)] = \log((\sqrt{\lambda_1} + \sqrt{\lambda_2})^2/2) - \gamma$

For $n=3$ I have the beginning of a series expansion: writing $\lambda_1 = \lambda_3(1 + \epsilon_1)$ and $\lambda_2 = \lambda_3 (1 + \epsilon_2)$,

$$ \eqalign{&E[\log(\lambda_1 Y_1^2 + \lambda_2 Y_2^2 + \lambda_3 Y_3^2)] = \cr &\ln \left({\lambda_3}/2 \right) +2-\gamma+ \frac{\epsilon_1 + \epsilon_2}{3} - \frac{3 \epsilon_1^2 + 2 \epsilon_1 \epsilon_2 + 3 \epsilon_2^2}{30} + \frac{5 \epsilon_1^3 + 3 \epsilon_1^2 \epsilon_2 + 3 \epsilon_1 \epsilon_2^2 + 5 \epsilon_2^3}{105} \cr &- \frac{\epsilon_1^4}{36} - \frac{\epsilon_1^3 \epsilon_2}{63} - \frac{\epsilon_1^2 \epsilon_2^2}{70} - \frac{\epsilon_1 \epsilon_2^3}{63} - \frac{\epsilon_2^4}{36} + \ldots\cr} $$

Hmm: it looks like for $\epsilon_2 = 0$ this might be

$$E[\log(\lambda_1 Y_1^2 + \lambda_3 Y_2^2 + \lambda_3 Y_3^2)] = \ln(\lambda_3/2) - \gamma + \ln(1 + \epsilon_1) + 2 \arctan(\sqrt{\epsilon_1})/\sqrt{\epsilon_1} $$

share|cite|improve this answer
Thank you for your help! – ppyang Nov 22 '11 at 2:16

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.