The answer is yes, but we need to assume here that $g$ be of bounded variation on $(A,B)$ (or on some other open neighborhood of $[a,b]$), rather than just on $[a,b]$. On the other hand, we only need to assume that $g$ is continuous at $a$ and $b$, rather than on the entire interval $[a,b]$ -- if the integrals here with respect to $dg$ are understood in the more general Lebesgue--Stieltjes sense.
Indeed, take any $h\in(b-a)\wedge(B-b)$. Let $g_+(x):=g(x+)$ for $x\in(A,B)$, so that $g_+$ is the right-continuous regularization of $g$. Then $g$ differs from $g_+$ only on a countable set of points, and so, the integral $\int_a^bf(x)\frac{g(x+h)-g(x)}h\,dx$ will not change if we replace there $g$ by $g_+$. So, without loss of generality $g=g_+$ and hence $g(x+h)-g(x)=\int_{(x,x+h]}dg(u)$ for all $x\in[a,b]$.
Hence, in view of the Fubini theorem,
\begin{equation}
\int_a^b f(x)[g(x+h)-g(x)]\,dx=\int_a^b f(x)\,dx\,\int_{(x,x+h]}dg(u)=I_1+I_{21}-I_{22}+I_3,
\end{equation}
where
\begin{align}
I_1&:=\int_{(a,a+h]}dg(u)\,\int_a^u f(x)\,dx = \int_{(a,a+h]}dg(u)\,O(h)=o(h), \\
I_{21}&:=\int_{(a,b]} dg(u)\,\int_{u-h}^u f(x)\,dx = \int_{(a,b]} dg(u)\,[hf(u)+o(h)]
=h\int_{(a,b]} dg(u)\,f(u)+o(h), \\
I_{22}&:=\int_{(a,a+h]} dg(u)\,\int_{u-h}^u f(x)\,dx = \int_{(a,a+h]} dg(u)\,O(h)
=o(h), \\
I_3&:=\int_{(b,b+h]} dg(u)\,\int_{u-h}^b f(x)\,dx = \int_{(b,b+h]} dg(u)\,O(h)
=o(h);
\end{align}
here we used the following: (i) $f$ is continuous on $(A,B)$ and hence bounded and uniformly continuous on some neighborhood of $[a,b]$ and (ii) $\int_{(c,c+h]}dg(u)=o(1)$ as $h\downarrow 0$, for each $c\in(A,B)$.
It follows that
\begin{equation}
\lim_{h\downarrow 0}\int_a^bf(x)\frac{g(x+h)-g(x)}h\,dx = \int_{(a,b]} dg(u)\,f(u).
\end{equation}
E.g., if $g=1_{[b,B)}$ or $g=1_{(b,B)}$, then
\begin{equation}
\lim_{h\downarrow 0}\int_a^bf(x)\frac{g(x+h)-g(x)}h\,dx =
\lim_{h\downarrow 0}\int_{b-h}^bf(x)\frac1h\,dx =f(b)= \int_{(a,b]} dg(u)\,f(u).
\end{equation}
Similarly (or just by using the reflection $x\mapsto-x$), we have
\begin{equation}
\lim_{h\uparrow 0}\int_a^bf(x)\frac{g(x+h)-g(x)}h\,dx = \int_{[a,b)} dg(u)\,f(u).
\end{equation}
So, if $g(a+)=g(a-)$ and $g(b+)=g(b-)$ (that is, if $g$ is continuous at $a$ and $b$), then
\begin{align*}
\lim_{h\to0}\int_a^bf(x)\frac{g(x+h)-g(x)}h\,dx &= \int_{[a,b]} dg(u)\,f(u)= \int_{(a,b)} dg(u)\,f(u) \\
& =\int_{(a,b]} dg(u)\,f(u)=\int_{(a,b]} dg(u)\,f(u).
\end{align*}