Complex manifolds in which the exponential map is holomorphic Let $X$ be a complex manifold and $g$ a hermitian metric on $X$. Consider the Riemannian exponential $\exp_p: T_p X \to X$.
If $\exp_p$ is holomorphic for every $p \in X$, then $(\exp_p)^{-1}$, suitably restricted, provide holomorphic normal coordinates near $p$, with respect to which the metric osculates to order 2 to the standard metric at the origin. This shows that $g$ is a Kähler metric.
However, Kähler is not sufficient to ensure that $\exp_p$ is holomorphic: take $X$ a curve of genus $g \geq 2$. If $\exp_p:T_pX \to X$ is holomorphic, then it lifts to a holomorphic map from $T_pX$ to the universal cover $\widetilde{X} = \Delta$, giving a holomorphic map $T_pX \simeq \mathbb{C} \to \Delta$, which must be constant by Liouville's theorem. In fact, one can see that $\exp$ cannot be holomorphic if $X$ is Kobayashi hyperbolic.
This leaves the question: What are the hermitian manifolds/metrics whose exponential map is holomorphic?
 A: I do not like complex numbers and can make a mistake easily...
Let $L_p$ be a complex line in a tangent space $T_pX$.
It is easy to see that $\exp_p$ gives an isometric embedding $L_p\hookrightarrow X$ which is also star-shaped with center at $p$; set $L=\exp_p(L_p)$
Take any other point $q\in L$, and let $L_q\subset T_q$ be the tangent subspace to $L$.
Note that the maps $\exp_p$ and $\exp_q$ coinside (up to a shift) on the geodesic $(pq)$.
[Here I use that if two holomorphic maps coincide on the real line then they coincide in the complex plane.]
It follows that $L=\exp_q(L_q)$.
Therefore $L$ is totally geodesic.
In other words:
For any complex sectional direction in $X$,
there is a tangent totally geodesic surface which is isometric to complex plane.
In particular, the sectional curvature in all complex sectional directions is zero
and therefore the curvature of $X$ is identically zero;
the later stated in Kobayashi--Nomizu, Foundations of differential geometry, Volume 2 IX, Prop. 7.1. (thanks to RdN).
A: NB:  I've had a little time to think about this and can now improve my answer, in particular, removing the real-analytic assumption, which, as I suspected, was not necessary.  Here is the improved answer:
If the metric $g$ is  Kähler, then having the exponential map from a point $p\in M$ be holomorphic makes it flat in a neighborhood of $p$.
Suppose that $\exp_p:T_pM\to M$ is holomorphic near $0_p\in T_pM$ (where we use the natural holomorphic structure on the complex vector space $T_pM$).  Let $z:T_pM\to\mathbb{C}^n$ be a complex linear isometry, so that the hermitian metric on $T_pM$ is just $|z|^2$ in the usual sense.  Let $Z$ be the holomorphic 'radial' vector field on $\mathbb{C}^n$, whose real part is the standard radial vector field on $\mathbb{C}^n$.  
Then
$$
{\exp_p}^*g = g_{i\bar j}(z)\ dz^i\ d\overline{z}^j
$$
for some functions $g_{i\bar j}$ on a neighborhood of $0\in\mathbb{C}^n$.  Since $g$ is Kähler, there is a function $f$ defined on a neighborhood of $0\in\mathbb{C}^n$ such that
$$
g_{i\bar j} = \frac{\partial^2f}{\partial z^i\ \partial\overline{z}^j}.
$$
Now, the condition that $z$ furnish Gauss normal coordinates for ${\exp_p}^*g$ is easily seen to be that
$$
\mathcal{L}_Z\bigl(\bar\partial f\bigr) = \bar\partial\bigl(|z|^2\bigr).
$$
In particular, $ \bar\partial\bigl(\mathcal{L}_Z(f - |z|^2)\bigr) = 0$, 
so $\mathcal{L}_Z(f - |z|^2) = h$ for some holomorphic function $h$ on a neighborhood of $0$.  This $h$ must vanish at $0$, so it is easy, by adding the real part of the appropriate holomorphic function to $f$ (which won't change $g$) to arrange that $h\equiv0$ and, moreover, that $f(0) = 0$.  But this now implies that the real-valued function $f-|z|^2$ vanishes at the origin and also is constant along the radial vector field.  Thus, $f = |z|^2$, and the metric $g$ is flat in these coordinates.
