The function $\vert x\vert^{\alpha-n}$ is radial homogeneous of degree $\alpha-n$, so its Fourier transform is radial homogeneous of degree $-(\alpha-n)-n=-\alpha$ (both locally integrable since $\alpha >0$ and $-\alpha>-n$ so both are distributions which are easily seen as temperate)


, so your convolution operator is in fact the Fourier multiplier $\vert D_x\vert^{-\alpha}$. The question at hand is thus (with homogeneous spaces)
$$
\Vert u\Vert_{W^{-\alpha,q}}\lesssim \Vert u\Vert_{W^{0,p}},\quad W^{0,p}\subset W^{-\alpha,q}
$$
which is a particular case of Sobolev injection since 
$$0>-\alpha,\quad
p < q,\quad \frac{1}{p}-\frac{1}{q}=\frac{\alpha}{n}
$$