You don't say what your motivating application is. It would be more effective to sample without replacement, but maybe that is not an option. You also suggest that you might be willing to limit the maximum number of times the most frequent item occurs. I think a lower bound on the least frequent color might be more helpful (equivalently, posit that anything below a certain relative frequency is an "anomaly" and does not count). I am just going to assume that a puzzle-master, unknown to us, has put $n$ colored balls in a bag according to some private scheme she made up. We sample $s$ times with replacement and must draw conclusions as best we can. I will assume that $n$ is known.
I think that you don't find a good match with species richness because you are not asking the right questions. What you can say something about is the possible frequency of the colors you have already seen. Indirectly that tells you something about the number of colors, but very little.
Do a thought experiment about various outcomes and what you can conclude. Suppose first that as you keep drawing (then replacing) balls they are all green. You can say that there seem to be lots of green balls. You can say "With $95\%$ confidence the number of green balls is at least $m=m(n,s)$" , equivalently the number of non-green balls is estimated to be no more than $n-m.$ However I do not think that you can make any conclusion about their colors. Even if you get a hint: " There are at least $q$ balls which are not green and they are either all the same color or else each has a color unique to itself" I do not think you have any way of saying which is more likely. And with no hint the only change is that perhaps $q=0$.
I don't think that a limit such as "no color occurs more than one third of the time" will help. Suppose instead that (for a not too small $s$) the balls come out "red, white, blue, yellow, pink" with frequencies roughly $2:2:2:1:1$. Then you can say "With $95\%$ confidence the number of red,white,blue,yellow and pink balls" balls is at least $m$" for the same $m=m(n,s)$ as in the all green scenario. AND you can say that whatever the number of balls of those five colors the frequencies will be roughly 1/4,1/4,1/4,1/8,1/8. But you are no better of on the question of how many unseen colors might there be.
I am less confident what one might say in the $2:2:2:1:1$ scenario above if along the way you say a single black ball, but I do not think that you could say much.
Another extreme is if you have made a reasonable number of samples and never seen the same color twice. Would that occur in your application? Then you might be able to say with some confidence that few, if any, colors repeat. Then you could (I would think) answer the "more than x colors" question as likely having seen many less than $x$ balls.