In terms of the differential operator $\partial_x\equiv d/dx$ one has $f(x+h)=e^{h\partial_x}f(x)$, hence
$$\Delta_h=e^{h\partial_x}-1.$$
Upon Fourier transformation, $\hat{f}(k)=\int_{-\infty}^\infty e^{ikx}f(x)\,dx$, one has $\partial_x\mapsto -ik$, hence
$$\hat{\Delta}_h=e^{-ihk}-1\Leftrightarrow \hat{\Delta}_h^{-1}=\frac{e^{ihk}}{1-e^{ihk}}.$$

In this way you can calculate, for any function $f(x)$ with Fourier transform $\hat{f}(k)$: 
$$\Delta_h^{-1}f(x)=\int_{-\infty}^\infty\frac{e^{i(h-x)k}}{1-e^{ihk}}\hat{f}(k)\,\frac{dk}{2\pi}=\int_{-\infty}^\infty F_h(x'-x)f(x')\,dx'$$
with <A HREF="https://www.google.com/search?client=safari&rls=en&q=dirac+delta+function&ie=UTF-8&oe=UTF-8">Dirac-delta-function</A> kernel
$$F_h(x)=\int_{-\infty}^\infty\frac{e^{i(h+x)k}}{1-e^{ihk}}\,\frac{dk}{2\pi}=\sum_{n=1}^\infty\delta(x+nh).$$
The relation $F_h(x)=(1/h)F_1(x/h)$ gives the scaling with $h$ requested by the OP: 

> $\Delta_h^{-1}f(x)$ follows from  $\Delta_1^{-1}f(rx)$ by rescaling
> $r\mapsto rh$ and $x\mapsto x/h$.

---

The "test cases" mentioned by the OP include: $\Delta_1^{-1}\log rx=x\log r+\log \Gamma(x)$, hence
$$\Delta_h^{-1}\log rx=(x/h)\log(rh)+\log\Gamma(x/h).$$
Similarly, $\Delta_1^{-1}a^{rx}=a^{rx}/(a^r-1)$, hence
$$\Delta_h^{-1} a^{rx}=a^{rx}/(a^{rh}-1),$$
and $\Delta_1^{-1}(rx)^a=r^aB_{a+1}(x)/(a+1)$, hence
$$\Delta_h^{-1}(rx)^a=(rh)^aB_{a+1}(x/h)/(a+1).$$