*The following is a rephrasing of and an expansion on Joris Bierkens' answer. It serves as my note for my own benefit. It would be a bonus if it helps any other people.*

Define stochastic processes $\displaystyle y_t := \exp\Big(-\int_0^t r_s \,ds \Big)$. Write $f(t) := \mathbb E [y_t]$. We apply the Ito's lemma recursively. In fact, we can derive the Ito version of the Taylor expansion to an arbitrary order.

Since $dr_tdy_t=-r_ty_tdr_tdt=0$, by Ito's Lemma
\begin{align}
-(y_t-y_0) &=\int_0^tr_sy_sds \\
&=\int_0^tds\Big(r_0y_0+\int_0^s(y_udr_u+r_udy_u)\Big) \\
&=r_0t+\int_0^tds\int_0^sy_u(\mu_u-r_u^2)\ du+\int_0^tdB_u\,y_u\sigma_u(t-u)
\end{align}
where the last integral comes from exchanging the order of two integrations.
Taking expectation
$$-(f(t)-1) = -\mathbb E \left[ y_t-1 \right] = r_0t+ \mathbb E \left[\int_0^t ds\int_0^s (\mu_u - r_u^2) y_u \ du  \right].$$
Interchanging integration and expectation, then differentiating with respect to $t$ gives
$$f'(t) = - r_0 - \int_0^t \mathbb E \left[ (\mu_u - r_u^2) y_u \right]du$$
and 
$$f''(t) = \mathbb E \left[ (r_t^2 - \mu_t) y_t \right].$$

Since the second derivative of $f(t)$ exists and is bounded with respect to $t$, it is bounded on $[0,T]$ for some $T>0$, 
$$f(t) = f(0)+f'(0)t+\frac{f''(\theta(t))}2t^2=1-r_0t+O(t^2)=\exp\big(-r_0t+O(t^2)\big)$$
for some $\theta(t)\in[0,t]$, as $t\downarrow 0$.

It remains to fill in details regarding the interchange of integration and expectation by Fubini's theorem, which should be justified using a priori estimates on the stochastic quantities using the strong linear growth assumptions.