As the $W$ matrix entries follow a multivariate normal distribution, then the probability that this matrix is singular is zero. 

That makes the matrix full rank, implying that $W^\dagger = W^{-1}$. That makes the last distance be zero: $|\mu- WW^\dagger\mu| = |\mu - I\mu| = 0$.

As Vincent stated in his comment, the interesting case is when $k<d$, and I realized that for that case the result changes as follows:

$|\mu-WW^\dagger\mu|=|(I-U\Sigma V^TV\Sigma^\dagger U^T)\mu|=|U(I-\Sigma\Sigma^\dagger)U^T\mu|$

Then the product $\Sigma\Sigma^\dagger$ will give us a $dxd$ matrix of the next form:

$\Sigma\Sigma^\dagger =  \left( \begin{array}{ccc}
I_{kxk} & 0\\
0 & 0\\
 \end{array} \right)
$

due to the fact that the matrix is full rank of rank k. That makes the next thing happen:
$\xi=(I - \Sigma\Sigma^\dagger) = \left( \begin{array}{ccc}
0 & 0\\
0 & I_{(d-k)x(d-k)}\\
 \end{array} \right)
$

so and as vector norm is unitarily invariant:

$|U\xi U^T\mu|=|\xi U^T \mu|$

and then=

$|\xi U^T \mu| = \left| \left( \begin{array}{ccc}
0 \\
u_{j>k}^T\\
 \end{array} \right)\mu\right|
=\left| \left( \begin{array}{ccc}
0 \\
u_{j>k}^T\mu\\
 \end{array} \right)\right| = 
\sum_{j>k}(u_j^T\mu)^2
$

So we get that value for the distance asked.