I know that you were mostly asking about cardinality, but more generally, you can ask about the structure of this space $\mathcal{M}(X,T)$ of invariant measures as a topological space with the weak topology (as Vaughn mentioned). It's known that $\mathcal{M}(X,T)$ must be a nonempty convex compact metrizable Choquet simplex, meaning that every point can be written as an "average" (integral) over extreme points. (This is because extreme points are precisely the ergodic measures, and the ergodic decomposition theorem states that every invariant measure is an "average" of ergodic measures).
In fact EVERY possible nonempty compact metrizable Choquet simplex is realizable as $\mathcal{M}(X,T)$ for some topological dynamical system $(X,T)$. There are all kinds of crazy Choquet simplices (in fact one where the extreme points are dense, as Vaughn mentioned), so you can get many different structures for the ergodic measures.
There are some hypotheses that guarantee that the set of ergodic measures is small. For instance, linear growth of the word complexity function implies this for subshifts. So does a uniform bound on the topological sequence entropy for all sequences. But the examples with huge $\mathcal{M}(X)$ are quite ubiquitous; for instance, there are several results showing that all possible Choquet simplices are realizable as spaces of invariant measures for topological systems from restricted classes (e.g. minimal systems, Toeplitz subshifts, logistic maps, etc.)