What is the canonical form of real symmetric 2n\times 2n matrix under unitary congruence? If $M$ is a $2n\times 2n$ real symmetric matrix, I would like to ask what could be its canonical form under unitary congruence. We view a unitary $n\times n$ matrix $U$ as a real $2n\times 2n$ matrix, and try to see if we can ``diagonalize'' $M$ by
$$U^T M U.$$
In addition, what is the canonical form if we only require that $U$ is symplectic?
 A: The difficulty is that the space $S_{2n}(\mathbb{R})$ of $2n$-by-$2n$ symmetric matrices with real entries is not irreducible under the action of $\mathrm{U}(n)$.  In fact, there is a $\mathrm{U}(n)$-irreducible decomposition
$$
S_{2n}(\mathbb{R}) = \mathbb{R}\cdot I_{2n}\oplus A_n\oplus B_n
$$
where $A_n$ is the space of matrices of the form
$$
A = \begin{pmatrix}s&a\cr-a&s\end{pmatrix}
$$
where $s$ is traceless symmetric $n$-by-$n$ and $a$ is antisymmetric $n$-by-$n$ and where $B_n$ is the space of matrices of the form
$$
B = \begin{pmatrix}p&q\cr q&-p\end{pmatrix}
$$
where $p$ and $q$ are $n$-by-$n$ symmetric matrices.  (In this realization, $\mathrm{U}(n)$ is the set of matrices of the form
$$
U = \begin{pmatrix}x&y\cr -y&x\end{pmatrix}
$$
where $x$ and $y$ are $n$-by-$n$ and $U = x+iy$ is unitary, i.e., $UU^\ast=I_n$.)
When you write $M = \lambda I_{2n} + A + B$ with $A\in A_n$ and $B\in B_n$, then you can use $U$ to diagonalize $A$ or you can use it to diagonalize $B$, but, usually, you cannot diagonalize both when $n>1$.
As for only requiring that $U$ be symplectic, now you are asking for a classification of the adjoint orbits of $\mathrm{Sp}(n,\mathbb{R})$ acting on its Lie algebra.  Of course, you can conjugate the semi-simple elements into a maximal torus (of some signature), but a full classification of, say, the nilpotent elements, is complicated.
