First, a purely mathematical remark: it is not so easy to construct Riemannian Ricci-flat metrics on compact manifolds. Ricci flat Kähler (= Calabi-Yau) metrics give a large class of examples and are "easy" to obtain: by Yau's theorem, it is enough to check a complex geometric condition (vanishing first Chern class) on a compact Kähler manifold to obtain a Ricci-flat metric. The point is that the equation Ricci curvature vanishes is easier to control with the additional Kähler requirement: a Kähler metric is locally determined by one complex valued function, whereas a general metric is locally determined by d(d-1)/2 real valued functions in (real) dimension d.
In physics, most of appearances of complex geometry are related in one way or the other to supersymmetry.
Given a supersymmetric theory on a Minkowski spacetime $\mathbb{R}^{1,D}$, you might want to consider the same theory on $\mathbb{R}^{1,d} \times M$, where $M$ is a compact manifold of dimension $D-d$. To obtain a supersymmetric theory on $\mathbb{R}^{1,d}$, the simplest possbility is to ask for $M$ to admit a covariantly constant spinor. Compact Riemannian manifolds admitting covariantly constant spinors are automatically Ricci-flat and necessarily have reduced holonomy. Calabi-Yau manifolds give a large class of such examples (but there are others, not necessarily obviously related to complex geometry, as 7-manifolds of G2 holonomy for example).
A "dual" point of view. Physically, it is natural to think to a particle probing spacetime and to study the worldline theory of this particle. For any Riemannian manifold, you can always consider a free particle, whose trajectories are geodesics. You can also consider a spinning particle, whose worldline theory is supersymmetric, and you can ask to which condition on the metric is the supersymmetry of the worldline theory bigger than expected. You find that Kähler manifolds are a natural class of examples where the worldline theory of the spinning particle has such an extended supersymmetry. (It is possible to go much further along these lines, e.g. replacing a point by a higher dimensional probing object as a string or a membrane and/or considering a quantum probing object, and to recover/interpret many facts about special holonomy metrics from such physics point of view).
