The following is a rephrasing of and an expansion on Joris Bierkens' answer. It serves as my note for my own benefit. It would be a bonus if it helps any other people.
Define stochastic processes $\displaystyle y_t := \exp\Big(-\int_0^t r_s \,ds \Big)$. Write $f(t) := \mathbb E [y_t]$. We apply Taylor's theorem to $f(t)$.
Since $dr_tdy_t=-r_ty_tdr_tdt=0$, by Ito's Lemma \begin{align} -(y_t-y_0) &=\int_0^tr_sy_sds \\ &=\int_0^tds\Big(r_0y_0+\int_0^s(y_udr_u+r_udy_u)\Big) \\ &=r_0t+\int_0^tds\int_0^sy_u(\mu(u,r_u)-r_u^2)\ du+\int_0^tdB_u\,y_u\sigma(u,r_u)(t-u) \end{align} where the last integral comes from exchanging the order of two integrations. Taking expectation $$-(f(t)-1) = -\mathbb E \left[ y_t-1 \right] = r_0t+ \mathbb E \left[\int_0^t ds\int_0^s (\mu (u,r_u) - r_u^2) y_u \ du \right].$$ Interchanging integration and expectation, then differentiating with respect to $t$ gives $$f'(t) = - r_0 - \int_0^t \mathbb E \left[ (\mu(u,r_u) - r_u^2) y_u \right]du$$ and $$f''(t) = \mathbb E \left[ (r_t^2 - \mu) y_t \right].$$
Since the second derivative of $f(t)$ exists and is bounded with respect to $t$, it is bounded on $[0,T]$ for some $T>0$, $$f(t) = f(0)+f'(0)t+\frac{f''(\theta(t))}2t^2=1-r_0t+O(t^2)=\exp\big(-r_0t+O(t^2)\big)$$ for some $\theta(t)\in[0,t]$, as $t\downarrow 0$.
It remains to fill in details regarding the interchange of integration and expectation by Fubini's theorem, which should be justified using a priori estimates on the stochastic quantities using the strong linear growth assumptions.