Suppose that $\mu=0$ --- for simplicity.
By Cauchy-Schwarz,
\begin{align*}
\mathbb{E} \left\{ \frac{ |X_{\infty}|}{\int_0^{\infty} f(X_s) ds} \right\} \le \sqrt{\mathbb{E} \left\{ X_{\infty}^2 \right\} \mathbb{E} \left\{ \left( \frac{1}{\int_0^{\infty} f(X_s) ds}\right)^2 \right\} } \;. \tag{1}
\end{align*}
Since $X$ is ergodic with non-normalized stationary density $e^{-\frac{x^2}{\sigma^2}} $, $$
\mathbb{E} \left\{ X_{\infty}^2 \right\} = \frac{\sigma^2}{2} \;, \tag{2}
$$
and , $$
\lim_{t \to \infty} \frac{1}{t} \int_0^t f(X_s) ds = \frac{1}{2} \left( 1 + \operatorname{erf}\left( \frac{a}{\sigma} \right) \right) \;.
\tag{3}
$$
Combining (1), (2) and (3) yields the desired result.