**Yes**, $g(t)$ is differentiable with respect to $t$ under the assumption that $a(s) > 0 $ for $s \in [0,t]$.

*Proof*. Introduce the time-change $\tau(s) := \int_0^s a(u)^{2} du$ and set $c(s) := b(\tau^{-1}(s)) a(\tau^{-1}(s))^{-2}$ for $s \in [0,t]$. In terms of this time change, the process $X_t$ can be written equivalently (in law) as $$
X_t \overset{d}{=} X_0 + \int_0^{\tau(t)} c(r) dr + W_{\tau(t)} \;.
$$ By Girsanov's Theorem, \begin{align*}
& g(t) = E\left[ 1(\inf_{0 \le s \le \tau(t)} \left( \int_0^s c(r) dr + W_s \right) \ge -X_0 ) \right] \\
&= E\left[ 1(\inf_{0 \le s \le \tau(t)} W_s \ge -X_0 ) e^{\left(\int_0^{\tau(t)} c(s) dW_s - \frac{1}{2} \int_0^{\tau(t)} c(s)^2 ds \right)} \right] \\
&= E\left[ 1(\sup_{0 \le s \le \tau(t)} (-W_s) \le X_0 ) e^{\left(\int_0^{\tau(t)} c(s) dW_s - \frac{1}{2} \int_0^{\tau(t)} c(s)^2 ds \right)} \right] \\
&= 1 - E\left[ 1(\sup_{0 \le s \le \tau(t)} W_s \ge X_0) e^{\left(-\int_0^{\tau(t)} c(s) dW_s - \frac{1}{2} \int_0^{\tau(t)} c(s)^2 ds \right)} \right]
\end{align*}
where in the last step we used that $-\inf_{0 \le s \le \tau(t)} W_s = \sup_{0 \le s \le \tau(t)} (-W_s)$ together with the reflection symmetry of Brownian motion, i.e., $-W_s$ is also a standard BM.

Let $T = \inf\{ t \ge 0 \mid W_t = X_0 \}$. By the strong Markov property of BM, \begin{equation}
g(t) = 1 - E\left[ 1(T \le \tau(t)) e^{\left(-\int_0^{T} c(s) dW_s - \frac{1}{2} \int_0^{T} c(s)^2 ds \right)} \right]
\tag{1} \label{g_T}
\end{equation}
Moreover, since the process $\{ W_s \}_{s \in [0,T]}$ is a Brownian bridge from $(0,0)$ to $(T,X_0)$, we have $$
W_s = X_0 \frac{s}{T} + (T-s) \int_0^s \frac{1}{T-r} dB_r \;,
$$ where $B$ is a standard BM. Hence, \begin{align*}
\int_0^{T} c(s) dW_s = \frac{X_0}{T} \int_0^T c(s) ds + \int_0^T \left( c(s) - \frac{1}{T-s} \int_s^T c(r) dr \right) d B_s
\end{align*}
Inserting this into \eqref{g_T} yields, \begin{align*}
&g(t) = \\
& \quad 1 - E\left[ 1(T \le \tau(t)) e^{- \int\limits_0^{T} \left(-\frac{X_0}{T} c(s)-\frac{1}{2} c(s)^2 + \frac{1}{2} (c_s - \frac{1}{T-s} \int\limits_s^T c_r dr)^2 \right) ds} \right] \end{align*} Since the distribution of the hitting time $T$ is inverse gamma with parameters $1/2$ and $(1/2) X_0^2$, \begin{equation} \begin{aligned}
&g(t) = \\
& \quad 1 - \int\limits_0^{\tau(t)} \frac{X_0 e^{- \frac{X_0^2}{2 z}- \int\limits_0^{z} \left(-\frac{X_0}{z} c(s)-\frac{1}{2} c(s)^2 + \frac{1}{2} (c_s - \frac{1}{z-s} \int\limits_s^z c_r dr)^2 \right) ds}}{\sqrt{2 \pi} z^{3/2}} dz
\end{aligned} \tag{$\star$} \label{final_g}\end{equation}
which is differentiable with respect to $t$ without any additional assumptions. $\Box$

This proof also answers this MO question: Probability that a drifted Gaussian process does not hit zero