Riesz transform of the function $f(x,y)$ of two variables ($d=2$),
$$ \mathcal{R}_x(f(x,y))=\frac{1}{2\pi}\iint \frac{(x-u)f(u,v)}{[(x-u)^2+(y-v)^2]^{3/2}}\,dudv.$$
Take the derivative with respect to $x$, make the desired substitutions
$$x\to -\frac{-x'-y'}{2 a},y\to -\frac{y'-x'}{2 b},$$
$$u\to -\frac{-u'-v'}{2 a},v\to -\frac{v'-u'}{2 b},$$
with Jacobian $2ab$, and you end up with
$$[\partial_x\mathcal{R}_xf](x',y')=\frac{2}{\pi}\iint \left(a^2(u'-v'-x'+y')^2-2b^2 (u'+v'-x'-y')^2\right)$$
$$\qquad{}\times\left(b^2(u'+v'-x'-y')^2+a^2(u'-v'-x'+y')^2\right)^{-5/2}f(u',v')\,du'dv'.$$
Any reason why you would want to go through this transformation?