The normalizer of SU(n) in U(m)?

Question

$\DeclareMathOperator\SU{SU}\DeclareMathOperator\U{U}\DeclareMathOperator\Spin{Spin}$Consider the special unitary group $\SU(5)$ and the unitary group $\U(16)$.

Below I specify a specfic way to embed $\SU(5) \subset \U(16)$:

1. First we can embed the spin group $\Spin(10)\subset \U(16)$. Here we choose the $\mathbf{16}$-dimensional spinor representation of $\Spin(10)$ to be also the $\mathbf{16}$-dimensional fundamental representation of $\U(16)$. Thus, the data for the representation: $$\text{$\mathbf{16}$ in $\Spin(10)$ as $\mathbf{16}$ in $\U(16)$}$$ gives us an action of $\Spin(10)$ and $\U(16)$ on the complex vector space $\mathbb{C}^{16}$.

2. Then, we can embed $\SU(5)$ into $\Spin(10)$. See the discussion of this embedding of $\SU(n)$ into $\Spin(2n)$. Here we choose the complex-conjugated fundamental representation $\overline{\mathbf 5}$, the anti-symmetric representation $\mathbf{10}$ and the 1-dimensional representation $\mathbf 1$ of $\SU(5)$: $$\text{$\overline{\mathbf 5} \oplus \mathbf{10} \oplus \mathbf 1$ in $\SU(5)$ as $\mathbf{16}$ in $\Spin(10)$}.$$ Again, the data for the representation gives us an action of $\SU(5)$ on $\mathbb{C}^{16}$, which further gives an embedding of $\SU(5)$ into $\U(16)$.

Question

Then my question is about the normalizer of this $\SU(5)$ inside $\U(16)$, which certainly depends on the embedding that I provided above. So what is the normalizer of this $\SU(5)$ inside $\U(16)$? My suspicion is that it may be $\U(5)$, but it could also be larger than $\U(5)$ with more $\U(1)$ factors if I was mistaken ….

Answer 1

$\DeclareMathOperator\U{U}\DeclareMathOperator\SU{SU}$It's $\U(5) \times \U(1) \times \U(1)$.

We have a natural map from the normalizer of $G$ to the outer automorphism group of $G$. The outer automorphism group of $G$ is $\mathbb Z/2$, generated by the inverse transpose / complex conjugation.

This map is trivial, because if it were nontrivial, an element mapping to $1 \in \mathbb Z/2$ would give an isomorphism between this representation and the complex conjugate $\mathbf 5 \oplus \overline{\mathbf {10}} \oplus \mathbf 1$, which does not exist as these are not isomorphic.

Let us check that the kernel of this natural map, for any $G \subseteq H$, is $G C_{H}(G)$, where $C_H(G)$ is the centralizer of $G$ in $H$. To do this, note that every element $h$ of the kernel acts by conjugation on $G$ as an inner automorphism. By the definition of inner automorphism, there exists $g \in G$ such that the action of $h$ by conjugation on $G$ equals the action of $g$ by conjugation on $G$. It follows that $g^{-1} h$ acts trivially by conjugation on $G$. By definition, this means $g^{-1} h$ lies in the centralizer of $G$. So $g = g \cdot (g^{-1}h) \in G C_H(G)$. Conversely, every element of $G C_H(G)$ normalizes $G$ and acts by inner automorphisms on $G$, since $C_H(G)$ acts trivially and $G$ acts by inner automorphisms.

The centralizer is $\U(1) \times \U(1) \times \U(1)$ since there are three non-isomorphic irreducible representations of $G$, so the centralizer of $G$ consists of matrices acting by scalars on these three representations.

So the normalizer is the product, inside $\U(16)$, of this $\SU(5)$ and the $\U(1) \times \U(1) \times \U(1)$ scalars. We can check that this product is isomorphic to $\U(5) \times \U(1) \times \U(1)$ by letting $\U(5)$ act on $\overline{\mathbf 5}$ by the dual of the standard representation, $\mathbf{10}$ by the $\wedge^2$ of the standard representation, and $\mathbf 1$ trivially, and letting the two factors of $\U(1)$ act by multiplication on the last two representations.