How does one prove that the splitting of primes in a nonabelian extension of number fields is not determined by congruence conditions?

Here's an answer for the special case when the base field is Q. It involves a large bit of class field theory over Q, so I'll be terse. We start with the lemma which Buzzard mentioned. Lemma  Let K, L be finite Galois extensions of Q. Then K is contained in L if and only if sp(L) is contained in sp(K) (with at most finitely many exceptions). The proof of the lemma follows from the Chebotarev Density Theorem. We now show that if the rational primes splitting in K can be described by congruences, then K/k is abelian. Proof. Assume that the rational primes splitting in K can be described by congruences modulo an integer a. This allows us to assume that Sp(K) contains the ray group P_a. The next step is to show that the rational primes lying in P_a are precisely the primes of sp($\Phi_a(x)$). By the above lemma, this means that K is contained in a cyclotomic field, hence is abelian. 


Fix a number field K. For an integer m, let S_1(m,K) be the congruence classes a mod m which contain infinitely many primes p such that pP for some prime P of K satisfying f(Pp) = 1. (That was a mouthful: p is lying below some prime of K with residue field degree 1.) If K/Q is Galois, then such p are the primes splitting completely in K, up to finitely many exceptions (among the ramified primes). That is, when K/Q is Galois, S_1(m,K) is the set of congruence classes mod m containing infinitely many primes which split completely in K. (The prime numbers which split completely in a number field are identical to the prime numbers which split completely in its Galois closure over Q, so attempting to describe such "split sets" by congruence conditions could just as well assume the number field is Galois over Q. I am working over base field Q throughout.) As Kevin has suggested, it is not obvious at first that these sets S_1(m,K) have much structure, particularly that they contain 1 mod m. By the pigeonhole principle, any S_1(m,K) is certainly a nonempty set, and it is a subset of the unit group (Z/m)* rather than just Z/m, but this is kind of superficial. A good reason (the right reason?) that 1 mod m is in S_1(m,K) is that S_1(m,K) is actually a subgroup of (Z/m)*. In fact, under the usual identification of Gal(Q(zeta_m)/Q) with (Z/m)* , S_1(m,K) is the image of the restriction homomorphism Gal(K(zeta_m)/K) > Gal(Q(zeta_m)/Q). For a proof, see Theorem 3 at http://www.math.uconn.edu/~kconrad/blurbs/gradnumthy/dirichleteuclid.pdf and Theorem 4 there is a generalization where (Z/m)* is replaced with any Galois group of number fields. 


OK how's about this to finish (I don't think either argument posted so far deals with this case). Say $K/\mathbf{Q}$ is finite and (away from a finite set of exceptions) $p$ splits completely in $K$ iff $p$ mod $N$ is contained in a subset $S$ of $(\mathbf{Z}/N\mathbf{Z})^\times$. I think the other two answers just deal with the case when $1\in S$ (where they show $K$ is contained in $\mathbf{Q}(\zeta_N)$). But if $1\not\in S$ then only a finite number of primes split completely in the compositum of the Galois closure of $K$ and $\mathbf{Q}(\zeta_N)$ and that's a contradiction. So now I think between us we have completely answered the question. 


A prime splits completely in $L$ over $K$ (an extension of number fields) if and only if it splits completely in the Galois closure of $L$ over $K$. Thus to answer the question we may assume that $L$ is Galois over $K$. Ben Linowitz's argument then holds in generality: suppose that all $\wp$ congruent to $1$ modulo some conductor $\mathfrak m$ split in $L$. Then by the Lemma in Ben's answer, $L$ is contained in the ray class field of conductor $\mathfrak m$ over $K$, and hence is abelian. (As far as I can tell, this is not at all obvious without class field theory, and in fact, a big part of the development of class field theory involved the realization that class fields  which were defined in terms of splitting conditiosn described by congruences  were the same things as abelian extensions. In some sense, the equivalence of these two conditions is the essence of class field theory.) [EDIT:] This edit is in response to Buzzard's comments on the original question, and also his answer and subsequent comments. Suppose that $L$ over $K$ is Galois (as we may) and that for some nonempty subset $S$ in some ray class group $Cl_{\mathfrak m}$ we know that all (but finitely many) primes lying in $S$ mod $\mathfrak m$ split in $L.$ If we furthermore assume if and only if in the preceding statement, then Buzzard's answer shows that $S$ must contain the trivial class, and hence $L$ is an abelian extension contained in the ray class field of conductor $\mathfrak m$; class field theory then takes over to show that $S$ is in fact a subgroup. But what if we don't assume if and only if (i.e. we allow that other primes besides those lying in $S$ split)? Can we still argue that $L$ is abelian over $K$? Let $L'$ be the compositum of $L$ and the ray class field of conductor $\mathfrak m$ over $K$, let $G = Gal(L/K)$, and let $G' = Gal(L'/K)$. Then $G' \hookrightarrow G \times Cl\_{\mathfrak m}$ (via the Galois action on $L'$ and the ray class field resp.); let $p$ and $q$ be the first and second projections (note that they are both surjective). Our assumption translates into the statement that $p (q^{1}(S)) = \{1\}$, i.e. $q^{1}(S) \subset 1 \times Cl\_{\mathfrak m}.$ Now choose $s \in S$, and suppose that $(g,1) \in G'$. The previous paragraph together with the surjectivity of $q$ shows that also $(1,s) \in G'$. Then $(g,1) (1,s) = (g,s) \in G',$ since $G'$ is a subgroup of the product. But $(g,s)$ lies in $q^{1}(S)$, hence $g = 1$. In other words, if the second coordinate of an element of $G'$ is trivial, so is the first. Thus in fact $L'$ equals the ray class field of conductor $\mathfrak m,$ i.e. $L$ is contained in the latter field. This is what had to be shown. 

