Let $K$ be a connected compact Lie group.  The moduli space of flat $K$-bundles over an $n$-torus is homeomorphic to the character variety $Hom(\mathbb{Z}^n,K)/K$.

The identity component of this space is homeomorphic to $T^n/W$ where $T$ is a maximal torus in $K$ and $W$ is the Weyl group (generalizing the example you gave when $K=SU(m)$).

Here are some examples for $K=SU(2)$:

1. $Hom(\mathbb{Z},K)/K=[-2,2]$

2. $Hom(\mathbb{Z}^2,K)/K=$[![Pillowcase][1]][1]

3.  $Hom(\mathbb{Z}^3,K)/K$ is a 3-dimensional orbifold.  Here is video of [a continuous family of slices of it.][2]

In general, the moduli space is connected iff $n=1$, or $n=2$ and $K$ is simply connected, or $n\geq 3$ and $K$ is isomorphic to a product of $SU(m)$'s and/or $Sp(k)$'s.  And the identity component of the moduli space is simply connected iff $K$ is semisimple (there may be other components that are not simply connected however).  Example 2. above shows that $\pi_2$ may not be trivial even if $K=SU(2)$.

Replacing $K$ with a complex (or real) reductive group $G$, we have a similar, although more complicated story.  

Please see my paper: *[Topology of character varieties of Abelian groups][3]* (co-authored with C. Florentino) for details about the reductive case (and references for the above mentioned facts).

Also, for the fundamental group of these moduli spaces please read my paper: *[Fundamental Groups of Character Varieties: Surfaces and Tori
][4]* (co-authored with I. Biswas and D. Ramras).


  [1]: https://i.sstatic.net/DEsrI.png
  [2]: https://youtu.be/wMj4OBq0HWo
  [3]: https://arxiv.org/abs/1301.7616
  [4]: https://arxiv.org/pdf/1412.4389.pdf