See Robert Bryant's answer to https://mathoverflow.net/questions/67903/complex-manifolds-in-which-the-exponential-map-is-holomorphic in which he proves that the surfaces for which the exponential map is conformal are precisely the flat ones.

If the exponential map is conformal on some Riemannian manifold of dimension $\ge 3$, it is a conformal map on the intersection $P \cap B$ of any 2-plane $P\subset T_m M$ in any tangent space with the ball $B\subset T_m M$ of radius equal to the injectivity radius. So the curvature vanishes on $e^{P\cap B}$, i.e. the sectional curvature vanishes, so the Riemannian manifold is flat.