# Average the covariance matrix over all orthogonal matrices

Let $$M=O\Lambda O^\top$$ be a positive semi-definite matrix, where $$\Lambda\in \mathbb{R}^{p\times p}$$ is a diagonal matrix with non-negative entries and $$O\in \mathbb{R}^{p\times p}$$ is an orthogonal matrix. Let $$S_p$$ be the set of all orthogonal matrices of size $$p\times p$$. What is the average of $$M$$ over the set $$S_p$$, i.e., what is the value of the following quantity? $$\begin{equation} \underset{O\in S_p}{\text{Average}}(M) \end{equation}$$

I presume you want to average over the orthogonal matrices uniformly, so with the Haar measure. Then $$\mathbb{E}[O_{ik}O_{jk}]=p^{-1}\delta_{ij}$$, hence $$\mathbb{E}[M_{ij}]=\mathbb{E}\left[\sum_{k=1}^p O_{ik}\lambda_k O_{jk}\right]=p^{-1}\delta_{ij}\sum_{k=1}^p\lambda_{k},$$ with $$\Lambda_{ij}=\delta_{ij}\lambda_i$$.
• may I ask why $E[O_{ik}O_{jk}]=p^{-1}\delta_{ij}$?
• this is the uniformity of the Haar measure; $O_{ik}$ and $-O_{ik}$ are equally probable, so you need $i=j$ for a nonzero answer, and then the answer can not depend on the index $k$, summing over $k$ gives 1, hence the factor $1/p$. May 11 at 21:01
• I know $E[\sum_{k=1}^p O_{ik}^2]=1$, but why $E[O_{ik}^2]=1/p$?
• the uniformity of the Haar measure tells you that the average of $O_{ik}^2$ is independent of $k$ (you can permute the index $k$ with some orthogonal permutation matrix $P$ and the Haar measure is invariant under $O\mapsto OP$). May 11 at 21:23