Powers of traces, integrals over spheres and class functions I asked this on math.StackExchange a while back but got no answers. I hope I'll be forgiven for the double post.
Let $V$ be a complex vector space of dimension $\operatorname{dim}_{\mathbb C} V = n$, equipped with a Hermitian inner product $\langle \,\cdot\,,\,\cdot\, \rangle$, and $A$ be an endomorphism of $V$. I am interested in calculating the functional
$$
\alpha_k(A) := \int_{S^{2n-1}} \langle Av, v \rangle^k \, d\mu,
$$
where $d\mu$ is the Lebesgue measure induced by the inner product and $S^{2n-1}$ is the unit sphere with respect to that product. For $k = 1$, this functional is a nonzero multiple of the trace, as shown in some older questions on math.SE. These kind of integrals pop up, for example, when we try to calculate the differential form that represents the $k$-th Segre class of a Hermitian holomorphic vector bundle.
Now, I don't really know any representation theory, but by extrapolating from one of the answers to the older questions the following should make a little sense: By polarizing we see that this functional is induced by the multilinear form
$$
\alpha_k(A_1, \ldots, A_k)
= \int_{S^{2n-1}} \langle A_1v, v \rangle \cdots \langle Av_k, v \rangle \, d\mu.
$$
This form is invariant under the action of $U(n) \times \cdots \times U(n)$ ($k$ times) on $\operatorname{Aut}(V^{\oplus k})$ (by conjugation on each factor), so $\alpha_k$ should be a class function on that representation. As such, it should be a linear combination of the characters of the representation. This should mean that $\alpha_k$ is a linear combination of the multilinear forms
$$
\def\tr{\operatorname{tr}}
\prod_{|J|=k} \tr(A_J),
$$
where we define $A_J = A_{j_1} \cdots A_{j_k}$ for a multiindex $J = (j_1,\ldots,j_k)$. For $k = 2$, for example, these are
$$
\tr(A_1^2), \quad \tr(A_1) \tr(A_2), \quad \tr(A_2)^2.
$$
Of course, $\alpha_k$ is a symmetric form, so there is some redundancy here ($\tr(A_1^2)$ and $\tr(A_2^2)$ must have the same coefficient in our hypothetical linear combination). 
We can calculate by brute force what happens in a special case here: Suppose $k = 2$ and $A$ is diagonalizable with eigenvalues $\lambda_1, \ldots, \lambda_n$. If $(e_1,\ldots,e_n)$ is an orthonormal basis of $V$ and $v = \sum z_j e_j$ a vector, we then have
$$
\langle Av,v \rangle = \sum_{j=1}^n \lambda_j |z_j|^2, \quad
\langle Av,v \rangle^2 = \sum_{j,k=1}^n \lambda_j \lambda_k |z_j|^2 |z_k|^2.
$$
Omitting some tedious calculations over the sphere, we then find that
$$
\eqalign{
\alpha_2(A) 
= \int_{S^{2n-1}} \langle Av,v \rangle^2 \,d\mu
&= \sum_{j,k=1}^n \lambda_j \lambda_k \int_{S^{2n-1}} |z_j|^2 |z_k|^2 \, d\mu
\cr
&= \frac{1}{n(n+1)}\sum_{j,k=1}^n \lambda_j \lambda_k 
+ \frac{1}{n(n+1)} \sum_{j=1}^n \lambda_j^2
\cr
&= \frac{1}{n(n+1)}( \tr(A)^2 + \tr(A^2) ),
}
$$
as we suspected. Unfortunately, this direct approach is completely hopeless for higher $k$, so we must look for other ways to calculate the integral.
Question Can we make my representation-theoretic hand-waiving precise to (1) show the existence of the hypothetical linear combination and (2) actually determine the coefficients in that combination?
 A: Begin by rewriting $\alpha_k(A_1,\ldots,A_k)$ as follows (just move integrals and traces around):
$$
\alpha_k(A_1,\ldots,A_k) = \mathrm{tr}\Big( (A_1 \otimes \cdots \otimes A_k) \int_{S^{2n-1}}(vv^* \otimes \cdots \otimes vv^*) d\mu \Big).
$$
Well, the integral on the right is well-known to be $P_{sym}/\binom{n+k-1}{n-1}$, where $P_{sym}$ is the projection onto the symmetric subspace (i.e., the subspace of $V \otimes \cdots \otimes V$ spanned by vectors of the form $v \otimes \cdots \otimes v$). We thus have
$$
\alpha_k(A_1,\ldots,A_k) = \frac{1}{\binom{n+k-1}{n-1}}\mathrm{tr}\big( (A_1 \otimes \cdots \otimes A_k) P_{sym} \big).
$$
One way to write $P_{sym}$ is as
$$
P_{sym} = \frac{1}{k!}\sum_{\pi\in S_k}W_{\pi},
$$
where $W_{\pi}$ is the unitary operator that permutes the $k$ tensor factors according to the permutation $\pi$ (and $S_k$ is the symmetric group on $k$ symbols).
Thus
$$
\alpha_k(A_1,\ldots,A_k) = \frac{1}{k!\binom{n+k-1}{n-1}}\sum_{\pi\in S_k}\mathrm{tr}\big( (A_1 \otimes \cdots \otimes A_k) W_{\pi} \big).
$$
Finally, the trace on the right can be computed in terms of traces of the $A_i$'s by breaking each $\pi$ into disjoint cycles and grouping the $A_i$'s that are within a common cycle within one trace. For example, if $\pi = (1 2)(3 4 5)$ (in cycle notation) then
$$
\mathrm{tr}\big( (A_1 \otimes \cdots \otimes A_5) W_{\pi} \big) = \mathrm{tr}(A_1 A_2)\mathrm{tr}(A_3 A_4 A_5),
$$
and hopefully it is clear how this generalizes.

When $k = 2$, this gives
$$
\alpha_2(A_1,A_2) = \frac{1}{n(n+1)}\big(\mathrm{tr}(A_1)\mathrm{tr}(A_2) + \mathrm{tr}(A_1A_2)\big),
$$
(the first term comes from the identity permutation, the second term comes from the transposition permutation) which agrees with the formula you found for the $k = 2$ case in your original post.

When $k = 3$, we get
$$
\alpha_3(A_1,A_2,A_3) = \frac{1}{n(n+1)(n+2)}\big(\mathrm{tr}(A_1)\mathrm{tr}(A_2)\mathrm{tr}(A_3) + \mathrm{tr}(A_1)\mathrm{tr}(A_2A_3) + \mathrm{tr}(A_2)\mathrm{tr}(A_1A_3)  + \mathrm{tr}(A_3)\mathrm{tr}(A_1A_2) + \mathrm{tr}(A_1A_2A_3) + \mathrm{tr}(A_3A_2A_1)\big).
$$
Again, we get one term in the sum for each of the $k! = 6$ permutations, and hopefully it is clear how they arise from the cycle decompositions of those permutations, and it generalizes to higher $k$ straightforwardly.
