Let $K$ be an imaginary quadratic field and $E$ be an elliptic curve with CM by $\mathcal{O}_K$. Is there a way to see that $K(j(E), h(E[\mathfrak{p}]))/K$ is an Abelian extension for some $\mathfrak{p}$ where $h$ is the Weber function and $j(E)$ is the $j$-invariant, without invoking the fact that it is the ray class field?

I am looking for a proof which is more elementary and probably along the lines of showing that $Gal(K(j(E), h(E[\mathfrak{p}]))/K)$ injects into an Abelian group, sort of like how we prove $K(j(E), E_{tors})/ K(j(E))$ is abelian.

Any proof I have found in literature involves showing that it is the ray class field of $K$ modulo $\mathfrak{p}$ using a characterization involving the splitting of primes.