The fact that you have to consider setiods shouldn't be *too* surprising, since if we're working ($n>0$)-categorically the distinction between sets and setoids isn't invariant under equivalence.  If C and D are strict 0-categories (i.e., sets), then the equivalence relation is just the identity relation.  If C and D are weak 0-categories (setoids), then $[C, D]$ is the weak 0-category (setoid) whose elements are the set functions $f : C \to D$, and where two functions $f, g: C \to D$ are equivalence if $f(c) \sim g(c)$ for all $c$ in $C$.

Sets equipped with equivalence relations have the simplest not-entirely-trivial homotopy theory I know.  [Larusson has a nice discussion of it.](https://arxiv.org/abs/math/0611344)