I always imagine the value of the multiplier $\lambda_i^*$ to tell me how badly one would like to violate the $i$-th constraint to **further** improve the objective function value from $x^*$.

Let us assume you have only *one* constraint, so you want to minimize a function $f$ over the feasible set given by $h(x) \leq 0$. Say we have a solution $x^*,\lambda^*$ to the KKT system, that is,

- $h(x^*) \leq 0$ (feasibility),
- $\nabla f(x^*) + \lambda^* \nabla
h(x^*) = 0$ (multiplier rule), and
- $\lambda^* \cdot h(x^*) = 0$ with
$\lambda^* \geq 0$ (complementarity),

and we further suppose that $x^*$ is actually a local minimum of the constrained problem.

So, if $h(x^*) < 0$, then the constraint clearly is not active at $x^*$ and since $x^*$ was already a local minimum, there can be no incentive (at least locally, but that's all general KKT theory can do) to move away from it because we were already free to do so.

Now, say $h(x^*) = 0$ and $\lambda^* > 0$. Then moving from $x^*$ in direction $\alpha\nabla h(x^*)$ for some scaling parameter $\alpha > 0$ will improve the objective value: $$f(x^* + \alpha \nabla h(x^*)) \approx f(x^*) + \alpha \nabla f(x^*)^T \nabla h(x^*) + r = f(x^*) - \alpha\lambda \|\nabla h(x^*)\|^2 + r$$ using the multiplier rule and Taylor expansion with some remainder term $r$, from which we can show that the left hand side is smaller than $f(x^*)$ for $\alpha$ small enough. Hence, violating the constraint given by $h$ would indeed give a better objective function value.

If $h(x^*) = \lambda^* = 0$, then $h$ is active at $x^*$, but there is nothing to be gained (again, locally), since $\nabla f(x^*)$ must already be zero by the multiplier rule! So, the constraint happens to be active, but not because one is eager to leave the feasible region towards higher values of $h$, but just "by accident".

Of course, these considerations become more complicated for more constraints which are active in $x^*$, but this should be a good starting point maybe.