The analogy is of course a correct/useful analogy, but I think any model structure for which the literal statement is correct must have $Set_* \simeq *$, so it would be a bit too coarse to do anything reasonable.
Indeed, your assumption guarantees the existence of a homotopy pullback square:
$$\require{AMScd} \begin{CD}Set_* @>>> * \\ @VVV @VVV \\ Set @>>> Set \end{CD}$$
But $Set \to Set$ is also a weak equivalence (it is an isomorphism !), so $$\require{AMScd} \begin{CD}* @>>> * \\ @VVV @VVV \\ Set @>>> Set \end{CD}$$ is also a homotopy pullback, which implies $Set_*\simeq *$.
Pseudo-limits (the kind where the diagrams commute up to isomorphism rather than just $2$-cells) can be expressed as homotopy limits, though. But for this kind of (op)lax limit, it seems like one need some notion of $2$-model category to make it work. I don't know more about this though.