I asked a similar question (note that there I called "$\mathcal{U}$-locally small categories" what you call "$\mathcal{U}$-categories"). I still don't have any strong opinion about this, but here are a few relevant points.
One generally wants to put some kind of bound on the objects of a locally small for the same reason one normally uses universes in category theory: in order to make the collection of "all" locally small categories into a legitimate 2-category, for example, so that one can talk about its relationship to other 2-categories.
The question is then: what bound? A natural alternative is to require that the collection of objects belongs to the successor universe of $\mathcal{U}$; or one could parameterize the definition on two universes.
A minor advantage to requiring the collection of objects of a category $C$ to be a subset of $\mathcal{U}$ is that, if you write "set" for "element of $\mathcal{U}$", then a "set of objects" of $C$ automatically has the correct meaning for settings in which the distinction between sets and classes of objects is important, like the theory of locally presentable categories. If the objects of $C$ don't have to belong to $\mathcal{U}$, then technically $C$ might not have any nonempty "sets" of elements in the above sense; in that case you may find yourself needing a word for "a set which is equinumerous to some element of $\mathcal{U}$"...
I used to believe something like your final parenthetical, but I think it is actually false. If $x$, $y \in \mathcal{U}$, then $(x, y) \in \mathcal{U}$; and then the key point is that if $A \in \mathcal{U}$ and $B \subset \mathcal{U}$, then each function $f : A \to B$ is actually an element of $\mathcal{U}$ under the standard encoding as a set of ordered pairs, because this set has the same cardinality as $A$ and each member $(a, f(a))$ belongs to $\mathcal{U}$. Then, functors from a $\mathcal{U}$-small category $C$ to a $\mathcal{U}$-category $D$ are also elements of $\mathcal{U}$, and so $[C, D]$ has as set of objects a subset of $\mathcal{U}$.
This argument is somewhat dependent on the encodings of ordered pairs and functions and you may not care for it much. In type-theoretic foundations, an argument like this may not even be possible; that is perhaps one philosophical reason to prefer the requirement that the collection of objects belongs to the successor universe of $\mathcal{U}$.
