Suppose that $\mu=0$ --- for simplicity.
By Cauchy-Schwarz, \begin{align*} \mathbb{E} \left\{ \frac{|X_t|}{\int_0^t f(X_s) ds} \right\} \le \sqrt{\mathbb{E} \left\{ X_t^2 \right\} \mathbb{E} \left\{ \left( \frac{1}{\int_0^t f(X_s) ds}\right)^2 \right\} } \;. \tag{1} \end{align*} A direct calculation shows that, $$ \mathbb{E} \left\{ X_t^2 \right\} = e^{-2 t} X_0^2 +\frac{\sigma^2}{2} (1-e^{-2 t}) \;, \tag{2} $$ and since $X$ is ergodic with stationary density proportional to $e^{-\frac{x^2}{\sigma^2}} $ , $$ \lim_{t \to \infty} \frac{1}{t} \int_0^t f(X_s) ds = \frac{1}{2} \left( 1 + \operatorname{erf}\left( \frac{a}{\sigma} \right) \right) \;. \tag{3} $$ Combining (1) and (2) yields, $$ \mathbb{E} \left\{ \frac{|X_t|}{\int_0^t f(X_s) ds} \right\} \le \frac{\sqrt{ X_0^2 + \frac{\sigma^2}{2} }}{t} \sqrt{\mathbb{E} \left\{ \frac{1}{ \left( \frac{1}{t} \int_0^t f(X_s) ds \right)^2} \right\} } $$ Passing to the limit as $t \to \infty$, applying (3) and DCT, gives the desired result.