Why do we care about small sets? I have been a user of category theory for a long time. I recently started studying a rigorous treatment of categories within ZFC+U. Then I become suspecting the effect of the smallness of sets.
We first fix notations.
Let $\mathbb{U}$ be a Grothendieck universe.
A set $x$ is called a $\mathbb{U}$-set if $x\in \mathbb{U}$.
A set is said to be $\mathbb{U}$-small if it is isomorphic to a $\mathbb{U}$-set.
We denote by $\operatorname{Set}_{\mathbb{U}}$ the category of $\mathbb{U}$-sets.
A category is called a locally $\mathbb{U}$-category (resp. locally $\mathbb{U}$-small) if its hom-sets are $\mathbb{U}$-sets (resp. $\mathbb{U}$-small).
There seem to be various advantages to a set being a $\mathbb{U}$-set.
For example,
$\mathbb{U}$-sets belong to $\operatorname{Set}_{\mathbb{U}}$ by the definition.
It is important when we construct the Yoneda embedding
$\mathcal{C} \to \operatorname{Fun}(\mathcal{C}^{\operatorname{op}},\operatorname{Set}_{\mathbb{U}})$
of locally $\mathbb{U}$-categories.
In contrast, we cannot construct such an embedding canonically for locally $\mathbb{U}$-small categories.
In various books, the fact that a set is small is treated as an important thing.
For example, let $\mathcal{A}$ be an abelian category.
Then its Grothendieck group $K_0(\mathcal{A})$ is defined by the free abelian group generated by the isomorphism classes $[X]$ of objects modulo the Euler relation:
$[B]=[A]+[C]$ if there exists a short exact sequence $0\to A \to B\to C\to 0$.
In many books (for example, Weibel`s K-book),
the skeletally $\mathbb{U}$-smallness is imposed on $\mathcal{A}$.
That is, suppose that
the set of isomorphism classes of objects of $\mathcal{A}$ is $\mathbb{U}$-small.
This guarantees that $K_0(\mathcal{A})$ is a $\mathbb{U}$-small abelian group.
However, how does this benefit us?
Note:
a category is a tuple $(\mathcal{C}_0,\mathcal{C}_1,s,t,e,\circ)$ of sets and maps satifying some conditions.
Thus I think that the Grothendieck group can be defined
without imposing any conditions on an abelian category $\mathcal{A}$.
 A: First, it is important to distinguish between the problem related to the foundation you are using from the problems that are inherent to category theory.
For example, the distinction between $\mathbb{U}$-small and $\mathbb{U}$-set is something that has to do with the set-theoretic foundation - in category theory, we don't consider properties that distinguish isomorphic objects so the notion of $\mathbb{U}$-set don't make sense (only $\mathbb{U}$-small).
Now, from the category-theoretic perspective, the only important thing is simply to keep track of what "size" are the objects you work with ("small" vs "large" though in some contexts one might need more than two sizes, "very large", etc...).
The problem isn't that you are not allowed to make some constructions - there are foundations that let you do pretty much everything you want with as many different sizes as you want - but only that you want to know in what categories the constructions you are making take values - do they produce small sets, large sets, "very large sets" etc... Some foundation might prevent you from doing some construction of course - but if you only focus on things that are "foundation independent" you can just change foundation if you run in this sort of problems - we do that very often.
To come back to your specific problem, It is completely fine to consider the $K_0$ of a "large" abelian category and get a "large" group. As pointed out by Andrej Brauer, in some foundations this might cause problems, but there definitely are foundations that can handle that sort of thing fine - for example the one you are talking about where everything is a set and size issues are handled with Grothendieck universe is indeed completely fine with this.
The thing is, if you build a large $K_0$, then you have a large group, but it is not an element of your "category of groups". If you want to make the $K_0$ construction into a functor you have to put some kind of size restriction on the category you apply it to. And if you want the category of "large group" to be a set, you are really going to need some kind of size restriction...
Now, there are many examples where not keeping track of size actually leads to problems. The most common is probably in the definition of limits and colimits: when one say that a category has all limits or colimits we always mean that it has all small limits or colimits. In fact it is a theorem that if a category has products (or coproduct) indexed by set of the same size as itself then it is a poset ! So when doing argument involving limits and colimits you always need to make sure the diagram you are taking limits and colimits on are small.
For example, the following argument is false because we are not careful with the size problems:
Fake Proposition: Every category with limits has an initial object.
Fake proof:  If $C$ has all limits then one can take the limit $L$ of the identity functor of $C \to C$. We will show that $L$ is an initial object. By construction, for every object $X \in C$ there is a map $f_X:L \to X$ and for every arrow $v:X \to Y$ we have $v f_X = f_Y$. Then for Every other arrow $k:L \to X$ we have $k f_L = f_X$. In particular, taking $k=f_X$ you get $f_X f_L = f_X$. It follows that $f_X f_L = f_X Id_L$ for all object $X$, and hence by the uniqueness part of the universal property of the limits $f_L = Id_L$, hence the equation above gives $k=f_X$ so there is a unique map $L \to X$ for each object $X$.
Note: The correct argument here of course show that if a category has limits indexed by itself then it has an initial object, but in traditional ZFC foundation, category having all limits of their own size are posets as mentioned above (There are however alternative foundation incompatible with ZFC where this result apply to categories that aren't posets).
Another situation is when building adjoint functors. It is pretty frequent that some would-be adjoint functors we want to exist actually don't because they take values in categories of "large objects" (for example large set or large group instead of the category of groups and sets) instead of the domain of the functor they are an adjoint of. This happens for example with the forgetful functor from complete boolean algebra to the category of sets - where the would-be left adjoint applied to any infinite set gives rise a something the size of the universe.
Of course in the case of $K_0$ you have two options: either only apply it to small categories, or you apply it to "large" categories, and consider that it takes values in the category of "large groups" - but then you need one more size because the category of "large groups" is itself "very large".
In the case of $K_0$ the reason why everybody makes the first choice and not the second is because in practice the $K_0$ construction is only interesting for categories like "finitely presented modules" or "finite-dimensional bundle/vector space" which are all essentially small categories.
As soon as you allow infinite dimensional objects in your category you end up with objects satisfying equation of the form $X \oplus X =X $ and $X \oplus Y = X$ for $Y$ finite-dimensional, which makes your $K_0$ group trivial. One can probably engineer interesting examples of large abelian categories with interesting $K_0$, but all the naive examples (like all groups, all bundles or all vector spaces) have trivial $K_0$ for this reason.
A: Given a class $C$ and an equivalence relation $\sim$ on $C$ we often wish to construct the quotient $C/{\sim}$. However, from the perspective of ZFC this is not such a straightforward things to do, because the elements of $C/{\sim}$ ought to be set-sized, but they may be proper classes. A proper class may not be an element of another class.
For instance, if $C$ is the category of abelian groups and $\sim$ is isomorphism, then the equivalence class of the trivial group would be a proper class because there are as many trivial groups as there are singleton sets.
The solution is to either curb the size of $C$ by requiring it to be $\mathbb{U}$-small, or to use Scott's trick: instead of working with the class-sized equivalence class $[x]_{\sim}$, we use as its representative the set $[x]_{\sim} \cap V_\alpha$, where $\alpha$ is the minimum rank of elements of $[x]_{\sim}$ and $V_\alpha$ is the $\alpha$-th level of the cummulative hierarchy.
In a category-theoretic setting, where notions of size appear in many situations, it is more practical to require $\mathbb{U}$-smallness than to rely heavily on the cummulative hierarchy (which may not even be available in some versions of category theory).
