The Riesz transform of the function $f(x,y)$ of two variables reads,
$$ \mathcal{R}_x(f(x,y))=\frac{1}{2\pi}\iint \frac{(x-u)f(u,v)}{[(x-u)^2+(y-v)^2]^{3/2}}\,dudv.$$
Let me define $F(x,y)=\partial_x\mathcal{R}_xf(x,y)$. The transformed functions $\tilde{f}(x',y')$ and $\tilde{F}(x',y')$ are defined by the coordinate transformation
$$x=-\frac{-x'-y'}{2 a},y=-\frac{y'-x'}{2 b},$$
with Jacobian $2ab$. Upon substitution you end up with
$$\tilde{F}(x',y')=\frac{2}{\pi}\iint \left(a^2(u'-v'-x'+y')^2-2b^2 (u'+v'-x'-y')^2\right)$$
$$\qquad{}\times\left(b^2(u'+v'-x'-y')^2+a^2(u'-v'-x'+y')^2\right)^{-5/2}\tilde{f}(u',v')\,du'dv'.$$
Any reason why you would want to go through this transformation?