Tyler, you are too fast: didn't give me a chance to answer first!
Of course, I agree with everything you say. I wrote the following
before seeing your answer (except for the last paragraph).

Since I introduced this choice, let me explain. But first, echoing
André, taking equivalence classes would be a wrong choice
even if it gave a category, which it doesn't. In fact, the word
“representation” in this context is a convenient lie:
we are not doing representation theory here, and we must not think
at all in terms of equivalence classes. For example, isomorphisms
between “representations” control signs in equivariant
cohomology theory.

One point is to obviate set theoretic nonsense. It has become
unfashionable, perhaps, to pay attention to this, but of course
the collection of all finite dimensional representations is not
a set, and for many purposes, such as taking colimits as you
say, one does want a set.

A mathematical point is that different universes give different
categories of G-spectra, and that matters enormously: change of
universe plays an essential role in equivariant stable homotopy
theory. This could be dealt with in other ways, but use of
universes is convenient.

Actually, how essential a universe is, depends on which choice
of a category of $G$-spectra one has in mind. For all choices,
it is very convenient to work with $G$-vector spaces with a
fixed given $G$-inner product. For orthogonal $G$-spectra, the fact
that the category $\mathcal I$ of such $G$-inner product spaces
is essentially small (equivalent to a small category) allows us to
use it without actually specifying a universe, although one does
obtain a different $\mathcal I$ for each choice of a set of irreducible
representations (the complete universe, allowing all, being the
most important).

For $G$-spectra in the sense of Gaunce Lewis and myself, and therefore
for the $S$-modules of EKMM (Elmendorf–Kriz–Mandell–May) use of a
universe is truly essential: $G$-spectra are obtained from $G$-prespectra
as colimits over inclusions of sub $G$-inner product spaces of a universe.
Such colimits make no sense without use of some device to ensure smallness.
In this line of development, use of a universe seems truly essential. The
linear isometries $G$-operad $\mathcal L$ is central to the construction of the
smash product (and to lots of work in equivariant infinite loop space
theory), and $\mathcal L(j)$ is the $G$-space of linear isometries $U^j\to U$,
where $U$ is the universe in which one is working. It would be ludicrous
to try to make sense of that without working in a universe.

As a philosophical point, it is essential to be eclectic in this area and to
allow use of different categories of $G$-spectra, such as orthogonal and Lewis–May
or EKMM, since there are many things that one can readily prove with one and not
the other. For a comparison of these two and discussion of change of universe, see
for example Mandell–May, Equivariant orthogonal spectra and $S$-modules.

That source explains how, in orthogonal $G$-spectra, one can actually work with one
fixed universe, even the trivial one, and obtain equivalent categories as Tyler
says. Hill–Hopkins–Ravenel took that observation from Mandell–May and ran with it.
To be honest, I sometimes regret we made that observation; as Tyler notes, it can
be a source of confusion, and it can sometimes obscure the mathematics. Here again
it is wise to be eclectic and think in terms of both physical change of universe
and “phantom” change of universe in terms of that observation.