Let $X \subseteq \mathbb{R}^n$ be compact and say the function $f \colon X \to \mathbb{R}$ is locally Lipschitz continuous. Say $\mathcal{X}$ is the set of all solutions of the following optimization problem: \begin{align} \min \quad & f(x) \\ \mathtt{s.t.} \quad x & \in X \end{align}
Are there any general results that establish the differentiability of $f$ at points in the set $\mathcal{X}$?
For example, I'm looking for conditions under which we can explicitly say that the function is not differentiable at any point in $\mathcal{X}$. A simple example of this is if the function $f$ is the absolute value function. Modifications to any of the assumptions made are allowed.