Let $X$ be a complex manifold and $g$ a hermitian metric on $X$. Consider the Riemannian exponential $\exp_p: T_p X \to X$.
If $\exp_p$ is holomorphic for every $p \in X$, then $(\exp_p)^{-1}$, suitably restricted, provide holomorphic normal coordinates near $p$, with respect to which the metric osculates to order 2 to the standard metric at the origin. This shows that $g$ is a Kähler metric.
However, Kähler is not sufficient to ensure that $\exp_p$ is holomorphic: take $X$ a curve of genus $g \geq 2$. If $\exp_p:T_pX \to X$ is holomorphic, then it lifts to a holomorphic map from $T_pX$ to the universal cover $\widetilde{X} = \Delta$, giving a holomorphic map $T_pX \simeq \mathbb{C} \to \Delta$, which must be constant by Liouville's theorem. In fact, one can see that $\exp$ cannot be holomorphic if $X$ is Kobayashi hyperbolic.
This leaves the question: What are the hermitian manifolds/metrics whose exponential map is holomorphic?