A prime splits completely in $L$ over $K$ (an extension of number fields) if and only if
it splits completely in the Galois closure of $L$ over $K$. Thus to answer the question we may assume that $L$ is Galois over $K$.  

Ben Linowitz's argument then holds in generality:  suppose that all $\wp$ congruent to $1$ modulo some conductor $\mathfrak m$ split in $L$.  Then by the Lemma in Ben's answer, $L$ is contained in the ray class field of conductor $\mathfrak m$ over $K$, and hence is abelian.

(As far as I can tell, this is not at all obvious without class field theory, and in fact, a big part of the development of class field theory involved the realization that class fields
--- which were defined in terms of splitting conditiosn described by congruences --- were the same things as abelian extensions.  In some sense, the equivalence of these two conditions is the essence of class field theory.)

[EDIT:] This edit is in response to Buzzard's comments on the original question, and also his answer and subsequent comments.

(Stupidity deleted; more to follow .. )