Return to Answer

let me work out the comments a bit further, starting from the identity (equation 8.2 from Diaconis and Evans, correcting an earlier paper by Diaconis and Shahshahani)

$$\int_{{\rm O}(n)}{\rm tr}\,g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

so the second integral of the OP evaluates to $$\int_{{\rm O}(n)}g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ n^{-1}\mathbb{1}&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

Taylor expansion of the exponent in the first integral of the OP and term-by-term integration gives $$ \int_{{\rm O}(n)}e^g\,dg=n^{-1}\mathbb{1}\sum_{p=0}^\infty \frac{1}{(2p)!}=\frac{\cosh 1}{n}\mathbb{1} $$

now for the third integral of the OP, we need the fourth-order tensor $$\int_{{\rm O}(n)}(g^p)_{ij}(g^q)_{kl}\,dg=a_{pq}(n)\delta_{ij}\delta_{kl}+b_{pq}(n)\delta_{ik}\delta_{jl}+c_{pq}(n)\delta_{il}\delta_{jk}$$ so that the required integral takes the form $$\int_{{\rm O}(n)}g^p Ag^q\,dg=a_{pq}(n)A+b_{pq}(n)A^{\rm t}+c_{pq}(n)\mathbb{1}\,{\rm tr}\,A $$ by taking traces I find, using again equation 8.2 from Diaconis and Evans, that $$na_{pq}(n)+nb_{pq}(n)+n^2 c_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{p+q}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ $$n^2a_{pq}(n)+nb_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}({\rm tr}\,g^p)({\rm tr}\,g^q)\,dg=\begin{cases} {\rm min}\,(p,2n)&{\rm if}\;\;p=q\;\;{\rm odd}\\ p+1&{\rm if}\;\;p=q\;\;{\rm even}\\ 1&{\rm if}\;\;p\neq q\;\;\text{both even}\\ 0&{\rm otherwise} \end{cases} $$ $$na_{pq}(n)+n^2 b_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{|p-q|}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ Three equations with three unknowns solves the third integral.

let me work out the comments a bit further, starting from the identity (equation 8.2 from Diaconis and Evans, correcting an earlier paper by Diaconis and Shahshahani)

$$\int_{{\rm O}(n)}{\rm tr}\,g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

so the second integral of the OP evaluates to $$\int_{{\rm O}(n)}g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ n^{-1}\mathbb{1}&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

Taylor expansion of the exponent in the first integral of the OP and term-by-term integration gives $$ \int_{{\rm O}(n)}e^g\,dg=n^{-1}\mathbb{1}\sum_{p=0}^\infty \frac{1}{(2p)!}=\frac{\cosh 1}{n}\mathbb{1} $$

now for the third integral of the OP, we need the fourth-order tensor $$\int_{{\rm O}(n)}(g^p)_{ij}(g^q)_{kl}\,dg=a_{pq}(n)\delta_{ij}\delta_{kl}+b_{pq}(n)\delta_{ik}\delta_{jl}+c_{pq}(n)\delta_{il}\delta_{jk}$$ so that the required integral takes the form $$\int_{{\rm O}(n)}g^p Ag^q\,dg=a_{pq}(n)A+b_{pq}(n)A^{\rm t}+c_{pq}(n)\mathbb{1}\,{\rm tr}\,A $$ by taking traces I find, using again equation 8.2 from Diaconis and Evans, that $$na_{pq}(n)+nb_{pq}(n)+n^2 c_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{p+q}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ $$n^2a_{pq}(n)+nb_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}({\rm tr}\,g^p)({\rm tr}\,g^q)\,dg=\begin{cases} {\rm min}\,(p,2n)&{\rm if}\;\;p=q\;\;{\rm odd}\\ {\rm min}\,(p,2n)+1&{\rm if}\;\;p=q\;\;{\rm even}\\ 1&{\rm if}\;\;p\neq q\;\;\text{both even}\\ 0&{\rm otherwise} \end{cases} $$ $$na_{pq}(n)+n^2 b_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{|p-q|}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ Three equations with three unknowns solves the third integral.

let me work out the comments a bit further, starting from the identity (theorem 4 from Diaconis and Shahshahani)

$$\int_{{\rm O}(n)}{\rm tr}\,g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

so the second integral of the OP evaluates to $$\int_{{\rm O}(n)}g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ n^{-1}\mathbb{1}&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

Taylor expansion of the exponent in the first integral of the OP and term-by-term integration gives \int_{{\rm O}(n)}e^g,dg=n^{-1}\mathbb{1}\sum_{p=0}^\infty \frac{1}{(2p)!}=\frac{\cosh 1}{n}\mathbb{1}
(incorrect, see bottom of post)

now for the third integral of the OP, we need the fourth-order tensor $$\int_{{\rm O}(n)}(g^p)_{ij}(g^q)_{kl}\,dg=a_{pq}(n)\delta_{ij}\delta_{kl}+b_{pq}(n)\delta_{ik}\delta_{jl}+c_{pq}(n)\delta_{il}\delta_{jk}$$ so that the required integral takes the form $$\int_{{\rm O}(n)}g^p Ag^q\,dg=a_{pq}(n)A+b_{pq}(n)A^{\rm t}+c_{pq}(n)\mathbb{1}\,{\rm tr}\,A $$ by taking traces I find, using again theorem 4 from Diaconis and Shahshahani, that $$na_{pq}(n)+nb_{pq}(n)+n^2 c_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{p+q}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ $$n^2a_{pq}(n)+nb_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}({\rm tr}\,g^p)({\rm tr}\,g^q)\,dg=\begin{cases} p&{\rm if}\;\;p=q\;\;{\rm odd}\\ p+1&{\rm if}\;\;p=q\;\;{\rm even}\\ 1&{\rm if}\;\;p\neq q\;\;\text{both even}\\ 0&{\rm otherwise} \end{cases} $$ $$na_{pq}(n)+n^2 b_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{|p-q|}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ Three equations with three unknowns give $$ a_{pq}=\frac{-2}{(n-1)(n+2)},\;\;b_{pq}=c_{pq}=\frac{n}{(n-1)(n+2)}\;\;\mbox{if $p\neq q$ both odd}$$ $$ a_{pq}=b_{pq}=c_{pq}=\frac{1}{n+2},\;\;\mbox{if $p\neq q$ both even}$$ $$a_{pq}=1-(n+1)b_{pq},\;\;b_{pq}=c_{pq}=\frac{n-p}{(n-1)(n+2)}\;\;\mbox{if $p=q$ odd}$$ $$a_{pq}=1-(n+1)b_{pq},\;\;b_{pq}=c_{pq}=\frac{n-p-1}{(n-1)(n+2)}\;\;\mbox{if $p=q$ even}$$

That should solve the third integral, but I may well have made some error, so the OP will want to check the algebra.

adddition/correction: there is a condition on $n$ that I have overlooked, because of a typo in Diaconis and Shahshahani (1994), eventually corrected in Diaconis and Evans (2001). This "theorem 4" requires that the sum $\kappa$ of the powers of $g$ in the integrand (in this case $\kappa=p$ or $\kappa=p+q$) should not be too large. There is some uncertainty on the optimal condition: $\kappa\leq n/2$ according to Diaconis and Evans, $\kappa\leq n-1$ according to Pastur and Vasilchuk, $\kappa\leq 2n$ according to this MO posting.

So this condition should be added to the above; the integral over $e^g$ involves arbitrarily high powers of $g$, hence it cannot be evaluated with the help of "theorem 4".

let me work out the comments a bit further, starting from the identity (equation 8.2 from Diaconis and Evans, correcting an earlier paper by Diaconis and Shahshahani)

$$\int_{{\rm O}(n)}{\rm tr}\,g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

so the second integral of the OP evaluates to $$\int_{{\rm O}(n)}g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ n^{-1}\mathbb{1}&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

Taylor expansion of the exponent in the first integral of the OP and term-by-term integration gives $$ \int_{{\rm O}(n)}e^g\,dg=n^{-1}\mathbb{1}\sum_{p=0}^\infty \frac{1}{(2p)!}=\frac{\cosh 1}{n}\mathbb{1} $$

now for the third integral of the OP, we need the fourth-order tensor $$\int_{{\rm O}(n)}(g^p)_{ij}(g^q)_{kl}\,dg=a_{pq}(n)\delta_{ij}\delta_{kl}+b_{pq}(n)\delta_{ik}\delta_{jl}+c_{pq}(n)\delta_{il}\delta_{jk}$$ so that the required integral takes the form $$\int_{{\rm O}(n)}g^p Ag^q\,dg=a_{pq}(n)A+b_{pq}(n)A^{\rm t}+c_{pq}(n)\mathbb{1}\,{\rm tr}\,A $$ by taking traces I find, using again equation 8.2 from Diaconis and Evans, that $$na_{pq}(n)+nb_{pq}(n)+n^2 c_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{p+q}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ $$n^2a_{pq}(n)+nb_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}({\rm tr}\,g^p)({\rm tr}\,g^q)\,dg=\begin{cases} {\rm min}\,(p,2n)&{\rm if}\;\;p=q\;\;{\rm odd}\\ p+1&{\rm if}\;\;p=q\;\;{\rm even}\\ 1&{\rm if}\;\;p\neq q\;\;\text{both even}\\ 0&{\rm otherwise} \end{cases} $$ $$na_{pq}(n)+n^2 b_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{|p-q|}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ Three equations with three unknowns solves the third integral.

let me work out the comments a bit further, starting from the identity (theorem 4 from Diaconis and Shahshahani)

$$\int_{{\rm O}(n)}{\rm tr}\,g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

so the second integral of the OP evaluates to $$\int_{{\rm O}(n)}g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ n^{-1}\mathbb{1}&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

Taylor expansion of the exponent in the first integral of the OP and term-by-term integration gives $$\int_{{\rm O}(n)}e^g\,dg=n^{-1}\mathbb{1}\sum_{p=0}^\infty \frac{1}{(2p)!}=\frac{\cosh 1}{n}\mathbb{1} $$

now for the third integral of the OP, we need the fourth-order tensor $$\int_{{\rm O}(n)}(g^p)_{ij}(g^q)_{kl}\,dg=a_{pq}(n)\delta_{ij}\delta_{kl}+b_{pq}(n)\delta_{ik}\delta_{jl}+c_{pq}(n)\delta_{il}\delta_{jk}$$ so that the required integral takes the form $$\int_{{\rm O}(n)}g^p Ag^q\,dg=a_{pq}(n)A+b_{pq}(n)A^{\rm t}+c_{pq}(n)\mathbb{1}\,{\rm tr}\,A $$ by taking traces I find, using again theorem 4 from Diaconis and Shahshahani, that $$na_{pq}(n)+nb_{pq}(n)+n^2 c_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{p+q}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ $$n^2a_{pq}(n)+nb_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}({\rm tr}\,g^p)({\rm tr}\,g^q)\,dg=\begin{cases} p&{\rm if}\;\;p=q\;\;{\rm odd}\\ p+1&{\rm if}\;\;p=q\;\;{\rm even}\\ 1&{\rm if}\;\;p\neq q\;\;\text{both even}\\ 0&{\rm otherwise} \end{cases} $$ $$na_{pq}(n)+n^2 b_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{|p-q|}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ Three equations with three unknowns give $$ a_{pq}=\frac{-2}{(n-1)(n+2)},\;\;b_{pq}=c_{pq}=\frac{n}{(n-1)(n+2)}\;\;\mbox{if $p\neq q$ both odd}$$ $$ a_{pq}=b_{pq}=c_{pq}=\frac{1}{n+2},\;\;\mbox{if $p\neq q$ both even}$$ $$a_{pq}=1-(n+1)b_{pq},\;\;b_{pq}=c_{pq}=\frac{n-p}{(n-1)(n+2)}\;\;\mbox{if $p=q$ odd}$$ $$a_{pq}=1-(n+1)b_{pq},\;\;b_{pq}=c_{pq}=\frac{n-p-1}{(n-1)(n+2)}\;\;\mbox{if $p=q$ even}$$

That should solve the third integral, but I may well have made some error, so the OP will want to check the algebra.

so my conclusion for the answer to the OP's question "when is $T_{nm}$ conjugate to $T_{n'm'}$" is: whenever $n\neq m$ and $n'\neq m'$ and $n+m+n'+m'$ even

let me work out the comments a bit further, starting from the identity (theorem 4 from Diaconis and Shahshahani)

$$\int_{{\rm O}(n)}{\rm tr}\,g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

so the second integral of the OP evaluates to $$\int_{{\rm O}(n)}g^p\,dg=\begin{cases} 0&{\rm if}\;\;p\;\;\text{is an odd integer}\\ n^{-1}\mathbb{1}&{\rm if}\;\;p\;\;\text{is an even integer} \end{cases} $$

Taylor expansion of the exponent in the first integral of the OP and term-by-term integration gives \int_{{\rm O}(n)}e^g,dg=n^{-1}\mathbb{1}\sum_{p=0}^\infty \frac{1}{(2p)!}=\frac{\cosh 1}{n}\mathbb{1}
(incorrect, see bottom of post)

now for the third integral of the OP, we need the fourth-order tensor $$\int_{{\rm O}(n)}(g^p)_{ij}(g^q)_{kl}\,dg=a_{pq}(n)\delta_{ij}\delta_{kl}+b_{pq}(n)\delta_{ik}\delta_{jl}+c_{pq}(n)\delta_{il}\delta_{jk}$$ so that the required integral takes the form $$\int_{{\rm O}(n)}g^p Ag^q\,dg=a_{pq}(n)A+b_{pq}(n)A^{\rm t}+c_{pq}(n)\mathbb{1}\,{\rm tr}\,A $$ by taking traces I find, using again theorem 4 from Diaconis and Shahshahani, that $$na_{pq}(n)+nb_{pq}(n)+n^2 c_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{p+q}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ $$n^2a_{pq}(n)+nb_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}({\rm tr}\,g^p)({\rm tr}\,g^q)\,dg=\begin{cases} p&{\rm if}\;\;p=q\;\;{\rm odd}\\ p+1&{\rm if}\;\;p=q\;\;{\rm even}\\ 1&{\rm if}\;\;p\neq q\;\;\text{both even}\\ 0&{\rm otherwise} \end{cases} $$ $$na_{pq}(n)+n^2 b_{pq}(n)+nc_{pq}(n)=\int_{{\rm O}(n)}{\rm tr}\,g^{|p-q|}\,dg=\begin{cases} 0&{\rm if}\;\;p+q\;\;\text{is an odd integer}\\ 1&{\rm if}\;\;p+q\;\;\text{is an even integer} \end{cases} $$ Three equations with three unknowns give $$ a_{pq}=\frac{-2}{(n-1)(n+2)},\;\;b_{pq}=c_{pq}=\frac{n}{(n-1)(n+2)}\;\;\mbox{if $p\neq q$ both odd}$$ $$ a_{pq}=b_{pq}=c_{pq}=\frac{1}{n+2},\;\;\mbox{if $p\neq q$ both even}$$ $$a_{pq}=1-(n+1)b_{pq},\;\;b_{pq}=c_{pq}=\frac{n-p}{(n-1)(n+2)}\;\;\mbox{if $p=q$ odd}$$ $$a_{pq}=1-(n+1)b_{pq},\;\;b_{pq}=c_{pq}=\frac{n-p-1}{(n-1)(n+2)}\;\;\mbox{if $p=q$ even}$$

That should solve the third integral, but I may well have made some error, so the OP will want to check the algebra.

adddition/correction: there is a condition on $n$ that I have overlooked, because of a typo in Diaconis and Shahshahani (1994), eventually corrected in Diaconis and Evans (2001). This "theorem 4" requires that the sum $\kappa$ of the powers of $g$ in the integrand (in this case $\kappa=p$ or $\kappa=p+q$) should not be too large. There is some uncertainty on the optimal condition: $\kappa\leq n/2$ according to Diaconis and Evans, $\kappa\leq n-1$ according to Pastur and Vasilchuk, $\kappa\leq 2n$ according to this MO posting.

So this condition should be added to the above; the integral over $e^g$ involves arbitrarily high powers of $g$, hence it cannot be evaluated with the help of "theorem 4".