I am interested in the distance between two conjugacy classes in a group like $SO(n)$. However let's consider $U(n)$ for simplicity. My conjecture is that the Hausdorff distance between the conjugacy classes of $\Lambda_1$ and $\Lambda_2$, both of which are diagonal matrices, is given by the distance of the eigenvalues $(\lambda_1, \ldots, \lambda_n)$ and $(\lambda^{(2)}_1, \ldots, \lambda^{(2)}_n)$ as elements in the homogeneous space $\mathbb{T}^n / S_n$, where the action of the symmetric group $S_n$ on $\mathbb{T}^n$ is by permutation of the coordinates. Thus by translation invariance, the closest element to $X$ in the conjugacy class of $Y$ is any one that can be simultaneously diagonalized with $X$. However I don't know how to prove this plausible statement.
Diagonalization gives a map $f\colon U(n)\to \mathbb{T}^n/S_n$. The map $f$ is also the projection to the orbitspace of of the $U(n)$action by conjugacy on $U(n)$. Hence $f$ is a submetry; i.e., $f(B_r(M))=B_r(f(M))$ for any matrix $M$. Hence your statement follows. 

