$\newcommand{\si}{\sigma}\newcommand{\vpi}{\varphi}\newcommand{\R}{\mathbb R}\newcommand{\De}{\Delta}$Let us show that the desired bound holds if
\begin{equation*}
\int_0^\infty dx\,|p'(x)|<\infty, \tag{1}\label{1}
\end{equation*}
which in particular implies that there exists the limit
\begin{equation*}
p_0:=p(0+)\in[0,\infty). \tag{2}\label{2}
\end{equation*}
One may note that, for condition \eqref{1} to hold, it is enough that e.g. the density $p$ be continuously differentiable and bounded on $(0,\infty)$ with only finitely many modes.
Let
\begin{equation*}
A:=A(y):=A(s,t,y),\quad B:=B(y):=B(s,t,y),
\end{equation*}
\begin{equation*}
h(x,y):=h(s,x,t,y):=f(s,x,t,y)-g(s,x,t,y),
\end{equation*}
\begin{equation*}
I:=\int_0^\infty dx\,p(x) \int_0^\infty dy\,h(x,y). \tag{3}\label{3}
\end{equation*}
We want to show that
\begin{equation*}
|I|\ll(t-s)^{1/2}\De a, \tag{$\clubsuit$}\label{*}
\end{equation*}
where $E\ll F$ means that $|E|\le cF$ for some real constant $c$ depending only on $p$ and
\begin{equation*}
\De a:=\|a-b\|_t.
\end{equation*}
Let $\Phi$ and $\vpi$ denote the standard normal cdf and pdf, respectively.
By \eqref{3} and \eqref{1},
\begin{equation*}
\begin{aligned}
I&=\int_0^\infty dx\,\Big(p_0+\int_0^x d\xi\,p'(\xi)\Big) \int_0^\infty dy\,h(x,y) \\
&=p_0 J+K,
\end{aligned}
\tag{4}\label{4}
\end{equation*}
where
\begin{equation*}
\begin{aligned}
J&:=\int_0^\infty dy\,\int_0^\infty dx\, h(x,y), \\
K&:=\int_0^\infty dx\,\int_0^x d\xi\,p'(\xi) \int_0^\infty dy\,h(x,y) \\
&{\color{red}{\,\,=}}\int_0^\infty dy\,\int_0^\infty d\xi\,p'(\xi)\int_\xi^\infty dx\, h(x,y) \\
&=\int_0^\infty dy\,\int_0^\infty d\xi\,p'(\xi)
\Big[\Phi\Big(\frac{y-\xi}{\sqrt{A(y)}}\Big)
-\Phi\Big(\frac{y-\xi}{\sqrt{B(y)}}\Big)\Big] \\
&=\int_0^\infty d\xi\,p'(\xi)
\int_0^\infty dy\,
\Big[\Phi\Big(\frac{y-\xi}{\sqrt{A(y)}}\Big)
-\Phi\Big(\frac{y-\xi}{\sqrt{B(y)}}\Big)\Big].
\end{aligned}
\tag{5}\label{5}
\end{equation*}
The red equality in \eqref{5} holds by the Fubini theorem -- which is the crucial point of the entire proof, as it allows one to deal, instead of $\big|\vpi\big(\frac z\si\big)'_\si\big|$, with $\big|\Phi\big(\frac z\si\big)'_\si\big|$ as in \eqref{!} below and thus get the crucial additional factor $|z|$ in the numerators there, which alleviates the possible smallness of the denominator $t-s$ of the ratio $\frac{2|z|}{t-s}$ in \eqref{!}.
Note that $\{A,B\}\subset[\frac{t-s}2,2(t-s)]$, and hence for any real $z$ and any $\si$ between $\sqrt A$ and $\sqrt B$ we have
\begin{equation*}
\Big|\Phi\Big(\frac z\si\Big)'_\si\Big|
=\frac{|z|}{\si^2}\,\vpi\Big(\frac z\si\Big)
\le\frac{2|z|}{t-s}\,\vpi\Big(\frac z{\sqrt{2(t-s)}}\Big) \tag{$\heartsuit$}\label{!}
\end{equation*}
and
\begin{equation*}
|\sqrt A-\sqrt B|=\frac{|A-B|}{\sqrt A+\sqrt B}\ll (t-s)^{1/2}\De a,
\end{equation*}
so that (by, say, the mean value theorem)
\begin{equation*}
\Big|\Phi\Big(\frac{y-\xi}{\sqrt{A(y)}}\Big)
-\Phi\Big(\frac{y-\xi}{\sqrt{B(y)}}\Big)\Big|
\ll\frac{|y-\xi|}{t-s}\,\vpi\Big(\frac{y-\xi}{\sqrt{2(t-s)}}\Big) (t-s)^{1/2}\De a
\end{equation*}
and
\begin{equation*}
\begin{aligned}
&\int_0^\infty dy\,
\Big|\Phi\Big(\frac{y-\xi}{\sqrt{A(y)}}\Big)
-\Phi\Big(\frac{y-\xi}{\sqrt{B(y)}}\Big)\Big| \\
&\ll\int_{-\infty}^\infty dy\,
\frac{|y-\xi|}{t-s}\,\vpi\Big(\frac{y-\xi}{\sqrt{2(t-s)}}\Big) (t-s)^{1/2}\De a \\
&\ll(t-s)^{1/2}\De a.
\end{aligned}
\end{equation*}
So, by \eqref{5} and \eqref{1},
\begin{equation*}
|K|\ll (t-s)^{1/2}\De a. \tag{6}\label{6}
\end{equation*}
Similarly and a bit easier, we get
\begin{equation*}
|J|\ll (t-s)^{1/2}\De a. \tag{7}\label{7}
\end{equation*}
Now \eqref{*} follows from \eqref{4}, \eqref{2}, \eqref{6}, and \eqref{7}.
