There are no Riemannian manifolds with differentiable metric $\rho$. Indeed, any Riemannian metric on the real line is locally isometric to the standard Euclidean metric with $\rho(x,y)=|x-y|$, which is not differentiable. On an arbitrary Riemannian manifold $M$ if $\gamma$ is a geodesic through $p$, then the restriction of $\exp_p$ to the tangent line to $\gamma$ is a a local isometry from $\mathbb R$ to the image of $\gamma$ (by Gauss lemma). So if $\rho$ were differentiable on $M$, the metric $|x-y|$ on the tangent line would be differentiable.
The best you can hope for is that the function $x\to\rho(x,p)$ is differentiable away from $p$. This happens exactly when $\exp_p$ is a diffeomorphism, e.g. it never happens when $M$ is compact.