Does the process of 'constructing the category of presheaves' always/never stabilize? Does it stabilize for some special class of categories?

That is, work in a foundation that allows for multiple levels of 'categorical largeness' and let $\mathcal{C}$ be a category. Recursively define

1. $\mathcal{C}_0=\mathcal{C}$,
2. $\mathcal{C}_{\alpha+1}={\bf Set}^{\mathcal{C}_\alpha^{op}}$,
3. $\mathcal{C}_\lambda=\varinjlim_{\alpha<\lambda}\mathcal{C}_\alpha$.

Does there always/never exist some $\alpha\in O_n$ such that $$\alpha\leq\beta\implies \mathcal{C}_\alpha\simeq\mathcal{C}_\beta?$$

I tried playing around with the case $\mathcal{C}={\bf Set}$ and it seems like we don't have the above property for finite $\alpha$, since ${\bf Set}$ is locally small and ${\bf Set}^{{\bf Set}^{op}}$ is not locally small (natural transformations are proper-class sized collections of sets), and higher finite iterations begin sending proper classes to sets etc. -- I don't know if this process stabilizes at $\omega$ or some higher limit ordinal, however.

Further, if we begin with the terminal category ${\bf 1}$ then ${\bf Set}^{{\bf 1}^{op}}\cong{\bf Set}$ so the above sequence repeats itself shifted back one step, and if we begin with the initial category ${\bf 0}$ then ${\bf Set}^{{\bf 0}^{op}}\cong{\bf 1}$, so the above sequence repeats itself shifted back two steps.

EDIT: In response to the confusion in Simon Henry's (otherwise excellent) answer, the Yoneda embedding plays no explicit role in the above question -- the colimit in the recursion is (a-priori) just over the discrete diagram on those categories. As he points out, if Yoneda was in play we would have to use the category of classes for a Yoneda embedding of ${\bf Set}^{{\bf Set}^{op}}$, and a category of 'superclasses' for the next one, so on and so forth. This isn't anything I'd imagined as part of the question, but if there are interesting answers to be had in this direction I'd like to hear them.