**Update and answer:** I asked a math professor at Princeton University. Sigh .. This answer is "No", and based on his explanation to me, hereunder is the quickest proof: suppose $d(x,y)$, $x \ne y$ and $x$ fixed, is differentiable at $y$, then $\vert \nabla d(x,y)\vert = 1$ has to hold. But when $y$ is a conjugate point, we have  $\nabla d(x,y) = 0$, a contradiction. In summary, $d(x,y)$ is differentiable at $y, y \ne x$ if and only if there is a unique minimal geodesic connecting $x$ and $y$ and $y$ is not a conjugate point to $x$, i.e., $d(x,y)$ is differentiable at $y, y \ne x$ if and only if $y$ is not in the cut locus of $x$. Thank you Leo Moos for your suggestion to put this as an answer.