What is the "no small subgroups" argument for $GL(n,\mathbb R)$? That is, how do we show that in $GL(n,\mathbb R)$ there exists a neighborhood of the identity which contains no subgroup other than the trivial one? I had some scribbling (for the $n=2$ case) but could not arrive at a clean proof.

7$\begingroup$ Just use the fact exponential map from the Lie algebra to the group is diffeomorphism on small neighbourhood of the identity and that $g^{k}=exp(k\cdot x)$ for some $x$ in the Lie algebra... $\endgroup$ – Asaf Aug 2 '12 at 15:21

$\begingroup$ @Asaf: I do not see how this suffices without more work. Can you fill in the rest of your argument? $\endgroup$ – Qiaochu Yuan Aug 3 '12 at 17:16
Here is Asaf's agrument expanded a bit. It has the advantage of working for all Lie groups simultaneously.
Given a Lie group $G$ with Lie algebra $\mathfrak{g}$, consider the exponential map $\exp:\mathfrak{g}\rightarrow G$. It is known that it is a diffeomorphism on a small enough open set $U\subseteq\mathfrak{g}$.
Choosing an inner product on $\mathfrak{g}$, we may assume wlog that $U$ has the form $U = \{v\in \mathfrak{g} : \; v < \epsilon\}$ for some $\epsilon > 0$. Let $V\subseteq U$ with $V = \{v\in\mathfrak{g} : \; v < \epsilon/2\}$.
I claim that $\exp(V)$ contains no nontrivial subgroups. Indeed, suppose $H\subseteq \exp(V)$ is a subgroup and choose $g\in H$ so $g = \exp(v)$ for some $v\in V$. I claim that $2v \in V$ as well. To see this, notice that since $g^2 \in H\subseteq \exp(V)$, we must have $g^2 = \exp(w)$ for some $w\in V$. Then $\exp(w) = g^2 = \exp(v)^2 = \exp(2v)$ which implies $w=2v$ since $\exp_U$ is a diffeomorphism. Thus, $2v \in V$.
But now can iterate this argument showing $2^n v \in V$ for all $n$. Since $2^n v = 2^n v$, this implies $v=0$, i.e. that $g =e$ so $H$ is trivial.

$\begingroup$ A typo: you are to claim that $2v\in V$, not $2v\in \exp(V)$. $\endgroup$ – Murat Güngör Aug 4 '12 at 12:14
It suffices to show that the powers of some nonidentity element $g \in \text{GL}_n(\mathbb{R})$ near the identity "escape from the identity." If $g$ has an eigenvalue not equal to $1$ then this follows by examining eigenvalues (we should take a neighborhood of the identity containing only elements with eigenvalues very close to $1$), so we reduce to the case that $g$ is unipotent. But now we can just compute that $1  g^k$ tends to $\infty$ in, say, HilbertSchmidt norm by writing $g$ as an uppertriangular matrix.
Edit: To show that a neighborhood containing elements with eigenvalues very close to $1$ exists, consider the neighborhood of elements whose characteristic polynomial is close to $(\lambda  1)^n$ (we will be more precise about this). Write $z = \lambda  1$, so we are trying to show that a polynomial of the form
$$z^n = a_{n1} z^{n1} + ... + a_0$$
has small roots if the $a_i$ are chosen to be small. Writing this as $1 = \frac{a_{n1}}{z} + ... + \frac{a_0}{z^n}$ we have
$$1 \le (a_{n1} + ... + a_0) \text{max} \left( \frac{1}{z}, \frac{1}{z^n} \right)$$
by the triangle inequality. We conclude that if we stipulate $a_{n1} + ... + a_0 < \text{min}(\epsilon, \epsilon^n)$ then $z < \epsilon$.

2$\begingroup$ Alternately, the closure of a small subgroup is compact, and a compact subgroup can be conjugated into $\text{O}(n)$ so has no unipotent elements. But this requires the existence of Haar measure... $\endgroup$ – Qiaochu Yuan Aug 2 '12 at 14:36

$\begingroup$ Still alternatively, every nontrivial subgroup of $O(n)$ with compact closure has an element whose trace is at most $n3/2$ (indeed, take a nonidentity element, take some power having eigenvalue $e^{ix}$ with $x\in [2\pi/3,4\pi/3]$). $\endgroup$ – YCor Aug 2 '12 at 17:25

$\begingroup$ Qiaochu, you say that 1 has a neighborhood consisting of matrices having eigenvalues close to 1; do you have a simple proof of this? $\endgroup$ – Murat Güngör Aug 3 '12 at 14:57


1$\begingroup$ It is more than a bit high powered, but one can also argue that the topology of "close roots" on the space of polynomials of at most a fixed degree makes it a finitedimensional topological vector space over a complete field, and there's only one of those in each dimension (and for each field); so that it must also be the same as the topology of "close coefficients". $\endgroup$ – LSpice Mar 24 '16 at 0:38