What is the maximal order of the automorphism group of a given Shimura variety?

Question

Background:

Given an elliptic curve $E$, it seems that $max(ord(Aut(E)))$ over the prime 2 is 24, and $(max(ord(Aut(E)))$ over the prime 3 is 12.

The endomorphism algebra of an elliptic curve over $\overline{\mathbb{F}}_p$ is, after tensoring with $\mathbb{Q}$, either an imaginary quadratic field or the unique division algebra $D_p$ which is non-split at $p$ and infinity. So, in order to compute the maximal order of the automorphism algebra, we just compute the units of $D_p$, and then count them.

---

Here's my question: Given a Shimura variety, how does one compute the maximal order of its automorphism group?

For example, the variety $\mathbb{C}^{\times 3}/ \mathbb{Z}[\zeta_7]^{\times 3}$ where the embedding is given by $\sigma_1 \times \sigma_2 \times \sigma_3: \mathbb{Z}[\zeta_7]^{\times 3} \to \mathbb{C}^{\times 3}$. Here, we look at its automorphisms over $\mathbb{F}_p$ where $p$ is $2$ or $4$ mod $7$.

I have heard that Shimura varieties have finite dimensional automorphism groups (analogous to the fact that $Aut(A, p)$ is finite, where $p$ is a polarization of the abelian variety $A$). What is the automorphism group of this Shimura variety?

Answer 1

Summary: The automorphism group of this variety at least contains $Z/2 \oplus Z/6 \simeq \mathbb{Z}[\zeta_7]^\times$,and is isomorphic to it if the variety is nondegenerate.

More generally, I learned today that given $A$ of the form $\mathbb{C}^{\phi(n)/2}/\mathbb{Z}[\zeta_n]$ (with the appropriate choice of embedding); then $Aut(A)$ is indeed simply $\mathbb{Z}[\zeta_n]^\times$ (and $End(A)$ is $\mathbb{Z}[\zeta_n]$.

To convince oneself that this is at the very least a subgroup of the endomorphism group, recall that endomorphisms of $\mathbb{C}^g/\Lambda$ are maps which preserve the lattice $\Lambda$ (or at least map the lattice to a scaled version of itself, such as $\Lambda \mapsto 2\Lambda$. For example, for any $x \in \mathbb{Z}[\zeta_n]$, we can multiply component-wise every point, taking the lattice to an $x$ scaled version of itself (where we consider $\zeta_n \in \mathbb{C}$ as $e^{2 \pi i/n}$):

$$\mathbb{C}^g \to \mathbb{C}^g$$

$$(p_1, ..., p_g) \mapsto (\sigma_1(x)p_1, ..., \sigma_g(x)p_g)$$