Suppose that $\mu=0$ --- for simplicity.  

By Cauchy-Schwarz, 
\begin{align*}
\mathbb{E} \left\{ \frac{|X_t|}{\int_0^t f(X_s) ds} \right\} \le \sqrt{\mathbb{E} \left\{ X_t^2 \right\} \mathbb{E} \left\{ \left( \frac{1}{\int_0^t f(X_s) ds}\right)^2 \right\}  } \;.  \tag{1}
\end{align*}
A direct calculation shows that, $$
\mathbb{E} \left\{ X_t^2 \right\} = e^{-2 t} X_0^2 +\frac{\sigma^2}{2} (1-e^{-2 t}) \;, \tag{2}
$$
and since $X$ is ergodic with stationary density proportional to $e^{-\frac{x^2}{\sigma^2}} $ , $$
\lim_{t \to \infty}  \frac{1}{t} \int_0^t f(X_s) ds  = \frac{1}{2} \left( 1 + \operatorname{erf}\left( \frac{a}{\sigma} \right) \right)  \;.
\tag{3}
$$
Combining (1) and (2) yields, $$
\mathbb{E} \left\{ \frac{|X_t|}{\int_0^t f(X_s) ds} \right\} \le \frac{\sqrt{ X_0^2 + \frac{\sigma^2}{2} }}{t}  \sqrt{\mathbb{E} \left\{ \frac{1}{ \left( \frac{1}{t} \int_0^t f(X_s) ds \right)^2} \right\}  } $$ 
Passing to the limit as $t \to \infty$, applying (3) and DCT, gives the desired result.