The short answer is that there are three positive-definite kinds of elementary inner products: 

1. symmetric on $\mathbb{R}^n$, giving rise to the orthogonal ensemble;
2. hermitian on $\mathbb{C}^n$, giving rise to the unitary ensemble; and
3. hermitian on $\mathbb{H}^n$, giving rise to the symplectic ensemble.

Each one gives rise to a compact classical Lie group: $\mathrm{O}(n)$, $\mathrm{U}(n)$ and $\mathrm{Sp}(n)$, respectively.  Compactness makes the integrals defining the matrix model convergent.