Let $M=O\Lambda O^\top$ be a positive semidefinite matrix, where $\Lambda\in \mathbb{R}^{p\times p}$ is a diagonal matrix with nonnegative entries and $O\in \mathbb{R}^{p\times p}$ is an orthogonal matrix. Let $S_p$ be the set of all orthogonal matrices of size $p\times p$. What is the average of $M$ over the set $S_p$, i.e., what is the value of the following quantity? \begin{equation} \underset{O\in S_p}{\text{Average}}(M) \end{equation}
I presume you want to average over the orthogonal matrices uniformly, so with the Haar measure. Then $\mathbb{E}[O_{ik}O_{jk}]=p^{1}\delta_{ij}$, hence $$\mathbb{E}[M_{ij}]=\mathbb{E}\left[\sum_{k=1}^p O_{ik}\lambda_k O_{jk}\right]=p^{1}\delta_{ij}\sum_{k=1}^p\lambda_{k},$$ with $\Lambda_{ij}=\delta_{ij}\lambda_i$.


3$\begingroup$ this is the uniformity of the Haar measure; $O_{ik}$ and $O_{ik}$ are equally probable, so you need $i=j$ for a nonzero answer, and then the answer can not depend on the index $k$, summing over $k$ gives 1, hence the factor $1/p$. $\endgroup$ May 11 at 21:01

$\begingroup$ I know $E[\sum_{k=1}^p O_{ik}^2]=1$, but why $E[O_{ik}^2]=1/p$? $\endgroup$– JohnMay 11 at 21:16

2$\begingroup$ the uniformity of the Haar measure tells you that the average of $O_{ik}^2$ is independent of $k$ (you can permute the index $k$ with some orthogonal permutation matrix $P$ and the Haar measure is invariant under $O\mapsto OP$). $\endgroup$ May 11 at 21:23