The function $\vert x\vert^{\alpha-n}$ is radial homogeneous of degree $\alpha-n$, so its Fourier transform is radial homogeneous of degree $-(\alpha-n)-n=-\alpha$ (both locally integrable since $\alpha >0$ and $-\alpha>-n$ so both are distributions which are easily seen as temperate: Fourier transforms make sense), so your convolution operator is in fact the Fourier multiplier $\vert D_x\vert^{-\alpha}$. The question at hand is thus (with homogeneous spaces)
$$
\Vert u\Vert_{W^{-\alpha,q}}\lesssim \Vert u\Vert_{W^{0,p}},\quad \text{i.e.  }W^{0,p}\subset W^{-\alpha,q},
$$
which is a particular case of Sobolev injection since 
$$0>-\alpha,\quad
p < q,\quad \frac{1}{p}-\frac{1}{q}=\frac{\alpha}{n}.
$$