The analogy is of course a correct/useful analogy, but I think any model structure for which the literal statement is correct must have $Set_* \simeq *$, so it would be a bit too coarse to do anything reasonable.

Indeed, your assumption guarantees the existence of a homotopy pullback square:

$$\require{AMScd} \begin{CD}Set_* @>>> * \\ @VVV @VVV \\ Set @>>> Set \end{CD}$$

But $Set \to Set$ is also a weak equivalence (it is an isomorphism !), so $$\require{AMScd} \begin{CD}* @>>> * \\ @VVV @VVV \\ Set @>>> Set \end{CD}$$ is also a homotopy pullback, which implies $Set_*\simeq *$.

Pseudo-limits (the kind where the diagrams commute up to isomorphism rather than just $2$-cells) can be expressed as homotopy limits, though. But for this kind of (op)lax limit, it seems like one need some notion of $2$-model category to make it work. I don't know more about this though.

The analogy is of course a correct/useful analogy, but I think any model structure for which the literal statement is correct must have $Set_* \simeq *$, so it would be a bit too coarse to do anything reasonable with categories as categories. As Alexander Campbell points out in the comments below, in the Thomason model stucture (which uses categories to model homotopy types) we do have $Set_* \simeq *$, but it destroys (what I guessed is) the point of you question.

Indeed, your assumption guarantees the existence of a homotopy pullback square:

$$\require{AMScd} \begin{CD}Set_* @>>> * \\ @VVV @VVV \\ Set @>>> Set \end{CD}$$

But $Set \to Set$ is also a weak equivalence (it is an isomorphism !), so $$\require{AMScd} \begin{CD}* @>>> * \\ @VVV @VVV \\ Set @>>> Set \end{CD}$$ is also a homotopy pullback, which implies $Set_*\simeq *$.

Pseudo-limits (the kind where the diagrams commute up to isomorphism rather than just $2$-cells) can be expressed as homotopy limits, though. But for this kind of (op)lax limit, it seems like one need some notion of $2$-model category to make it work. I don't know more about this though.