Yes. Let $\gamma$ be the geodesic from $x$ to $y$. The defining equation $\nabla_{\gamma^\prime}\gamma^\prime=0$ implies that parallel transport along $\gamma$ maps the tangent vector $\gamma^\prime_x$ to the tangent vector $\gamma^\prime_y$. But the first is a multiple of $log_yx$$\log_yx$ and the second is a multiple of $-log_xy$$-\log_xy$. Since parallel transport preserves length of vectors, and since $\parallel log_xy\parallel=d(x,y)=d(y,x)=\parallel log_yx\parallel$$\|\log_xy\|=d(x,y)=d(y,x)=\|\log_yx\|$, your claim follows.