Let $G$ be a connected Lie group. Equivalences:
- (i) every linear representation of $G$ has a closed image
- (ii) $\mathrm{Hom}(G,\mathbf{R})=0$
- (iii) $G/\overline{[G,G]}$ is compact.
Here homomorphisms are meant continuous, and reps are in $\mathbf{GL}_n(\mathbf{R})$ for $n$ not fixed, or equivalently in $\mathrm{GL}_n(\mathbf{C})$ in view of the closed inclusions $\mathrm{GL}_n(\mathbf{R})\subset \mathrm{GL}_n(\mathbf{C})\subset \mathrm{GL}_{2n}(\mathbf{R})$. Note that the equivalence between (ii) and (iii) is obvious.
That (i) implies (ii) is clear since if $\mathrm{Hom}(G,\mathbf{R})\neq 0$, $G$ admits $\mathbf{R}$ as a quotient, and we can use a 2-dimensional complex representation of $\mathbf{R}$ whose image is dense in a 2-dimensional real torus.
The implication (ii)$\Rightarrow$(i) goes by reduction to the semisimple case.
Thus first assume that $G$ is semisimple. Fix a representation $\rho:G\to\mathrm{GL}_n(\mathbf{C})$. In the latter, the image of the representation is a (a priori not closed) Lie subgroup $H_1$, with Lie algebra $\mathfrak{h}$ (a real Lie aubalgebra of $\mathfrak{gl}_n(\mathbf{C})$.
Let $H_2$ be the closure of $H_1$ (in the real Zariski topology, or in the real topology, it does not matter). Then $H_2$ normalizes $\mathfrak{h}$ and $H_1$. Since up to finite index, the automorphisms of $H_1$ are inner, and since $H_2$ is connected, we deduce that $H_2=H_1C$ where $C$ is the centralizer of $H_1$ in $H_2$. Decompose $\mathbf{C}^n=\bigoplus V_i$, a sub of $H_1$-irreducible complex subspaces. Then since $C$ is in the closure of $H_1$, it preserve each $V_i$. Since the $H_1$-representation on the complex space $V_i$ is irreducible, its commutant is reduced to scalars. Moreover, since the restriction of $H_1$ on $V_i$ has determinant 1, it is also the case for $C$. We deduce that $C$ is a finite abelian group. Thus $H_1$ has finite index in $H_2$. Since $H_1$ is $\sigma$-compact, a Baire argument then shows that $H_1$ has non-empty interior in $H_2$, and hence $H_1$ is the unit component of $H_2$ in the real topology. Thus $H_1$ is closed.
(Another argument goes by using the fact that perfect Lie subalgebras of $\mathfrak{gl}_n(\mathbf{R})$ are always Lie algebras of some Zariski closed subgroup of $\mathrm{GL}_n(\mathbf{R})$.)
Now assume $G$ arbitrary satisfying (ii). Write $G=RS$ with $R$ the connected solvable radical and $S$ a semisimple Levi factor. A simple argument
based on taking the complexification at the Lie algebra level shows that $[G,R]$ is normal and that each representation of $G$ maps $[G,R]$ to unipotents. Thus let $\rho$ be a representation of $G$, say in $\mathrm{GL}_n(\mathbf{R})$, and let $H$ be its image. Then $N=\rho\Big(\overline{[G,R]}\Big)$ is a connected, unipotent normal subgroup of $H$. Being a unipotent connected Lie subgroup, it is necessarily closed in $\mathrm{GL}_n(\mathbf{R})$, actually Zariski closed. If $M$ is the normalizer of $N$, then $H\subset M\subset \mathrm{GL}_n(\mathbf{R})$ and $M$ is Zariski closed. Moreover $M/N$ stands a Zariski closed subgroup of $\mathrm{GL}_m(\mathbf{R})$ for some $m$ (indeed, if $M=\mathbb{M}_\mathbf{R}$ and $N=\mathbb{N}_\mathbf{R}$ for some $\mathbf{R}$-subgroups of $\mathrm{GL}_n$, then $\mathbb{M}/\mathbb{N}$ is a linear algebraic group and hence is isomorphic to an $\mathbf{R}$-closed subgroup in some $\mathrm{GL}_m$, and $\mathbb{M}_\mathbf{R}/\mathbb{N}_\mathbf{R}\to (\mathbb{M}/\mathbb{N})_\mathbf{R}$ is a closed map in the real topology.)
Therefore, we are reduced to the case when $[G,R]=1$. In this case $G=RS$, with $S$ semisimple Levi factor, $R$ abelian, and $[R,S]=1$. Since every linear representation of $S$ factors through some finite index subgroup of its center, it is no restriction to assume that $S$ has a finite center, and in particular $S$ is closed. By (ii), we thus deduce that $R$ is compact. Hence since by the first case, $\rho(S)$ is closed, we deduce that $\rho(G)$ is closed as well.