Why do congruence conditions not suffice to determine which primes split in non-abelian extensions? How does one prove that the splitting of primes in a non-abelian extension of number fields is not determined by congruence conditions?
 A: Here's an answer for the special case when the base field is $\mathbb{Q}$. It involves a large bit of class field theory over $\mathbb{Q}$, so I'll be terse.
We start with the lemma which Buzzard mentioned.
Lemma - Let $K$, $L$ be finite Galois extensions of $\mathbb{Q}$. Then $K$ is contained in $L$ if and only if $\operatorname{sp}(L)$ is contained in $\operatorname{sp}(K)$ (with at most finitely many exceptions).
The proof of the lemma follows from the Chebotarev Density Theorem.
We now show that if the rational primes splitting in $K$ can be described by congruences, then $K/k$ is abelian.
Proof. Assume that the rational primes splitting in $K$ can be described by congruences modulo an integer $a$. This allows us to assume that $\operatorname{Sp}(K)$ contains the ray group $P_a$. The next step is to show that the rational primes lying in $P_a$ are precisely the primes of $\operatorname{sp}(\Phi_a(x))$. By the above lemma, this means that $K$ is contained in a cyclotomic field, hence is abelian.
A: OK how's about this to finish (I don't think either argument posted so far deals with this case). Say $K/\mathbf{Q}$ is finite and (away from a finite set of exceptions) $p$ splits completely in $K$ iff $p$ mod $N$ is contained in a subset $S$ of $(\mathbf{Z}/N\mathbf{Z})^\times$. I think the other two answers just deal with the case when $1\in S$ (where they show $K$ is contained in $\mathbf{Q}(\zeta_N)$). But if $1\not\in S$ then only a finite number of primes split completely in the compositum of the Galois closure of $K$ and $\mathbf{Q}(\zeta_N)$ and that's a contradiction. So now I think between us we have completely answered the question.
A: Fix a number field $K$.  For an integer $m$, let $S_1(m,K)$ be the congruence classes $a$ mod $m$ that contain infinitely many primes $p$ such that $p \mid \mathfrak p$ for some prime $\mathfrak p$ of $K$ satisfying $f(\mathfrak p|p) = 1$. (That was a mouthful: $p$ is lying below some prime of $K$ with residue field degree 1.) 
If $K/\mathbf Q$ is Galois, then such $p$ are the primes splitting completely in $K$, up to finitely many exceptions (among the ramified primes).  That is, when $K/\mathbf Q$ is Galois, $S_1(m,K)$ is the set of congruence classes mod $m$ containing infinitely many primes which split completely in $K$. (The prime numbers that split completely in a number field are identical to the prime numbers that split completely in its Galois closure over $\mathbf Q$, so attempting to describe such "split sets" by congruence conditions could just as well assume the number field is Galois over $\mathbf Q$.  I am working over base field $\mathbf Q$ throughout.)
As Kevin has suggested, it is not obvious at first that these sets $S_1(m,K)$ have much structure, particularly that they contain $1$ mod $m$.  By the pigeonhole principle, any $S_1(m,K)$ is certainly a nonempty set, and it is a subset of the unit group $(\mathbf Z/m)^\times$ rather than just $\mathbf Z/m$, but this is kind of superficial.  
A good reason (the right reason?) that $1$ mod $m$ is in $S_1(m,K)$ is that $S_1(m,K)$ is actually a subgroup of $(\mathbf Z/m)^\times$.  In fact, under the usual identification of $\mathrm{Gal}(\mathbf Q(\zeta_m)/ \mathbf Q)$ with $(\mathbf Z/m)^\times$, $S_1(m,K)$ is the image of the restriction homomorphism $\mathrm{Gal}(K(\zeta_m)/K) \longrightarrow \mathrm{Gal}(\mathbf Q(\zeta_m)/\mathbf Q)$.  For a proof, see Theorem 3 at 
http://www.math.uconn.edu/~kconrad/blurbs/gradnumthy/dirichleteuclid.pdf
and Theorem 4 there is a generalization where $(\mathbf Z/m)^\times$ is replaced with any Galois group of number fields. 
A: A prime splits completely in $L$ over $K$ (an extension of number fields) if and only if
it splits completely in the Galois closure of $L$ over $K$. Thus to answer the question we may assume that $L$ is Galois over $K$.  
Ben Linowitz's argument then holds in generality:  suppose that all $\wp$ congruent to $1$ modulo some conductor $\mathfrak m$ split in $L$.  Then by the Lemma in Ben's answer, $L$ is contained in the ray class field of conductor $\mathfrak m$ over $K$, and hence is abelian.
(As far as I can tell, this is not at all obvious without class field theory, and in fact, a big part of the development of class field theory involved the realization that class fields
--- which were defined in terms of splitting conditiosn described by congruences --- were the same things as abelian extensions.  In some sense, the equivalence of these two conditions is the essence of class field theory.)
[EDIT:] This edit is in response to Buzzard's comments on the original question, and also his answer and subsequent comments.
Suppose that $L$ over $K$ is Galois (as we may) and that for some non-empty subset $S$ in some ray
class group $Cl_{\mathfrak m}$ we know that all (but finitely many) primes lying in $S$ mod $\mathfrak m$ split in $L.$  
If we furthermore assume if and only if in the preceding statement, then Buzzard's answer shows that $S$ must contain the trivial class, and hence $L$ is an abelian extension contained in the ray class field of conductor $\mathfrak m$; class field theory then takes over to show that $S$ is in fact a subgroup.
But what if we don't assume if and only if (i.e. we allow that other primes besides those
lying in $S$ split)?  Can we still argue that $L$ is abelian over $K$?
Let $L'$ be the compositum of $L$ and the ray class field of conductor $\mathfrak m$
over $K$, let $G = Gal(L/K)$, and let $G' = Gal(L'/K)$.  
Then $G' \hookrightarrow G \times Cl\_{\mathfrak m}$ (via the Galois action on $L'$
and the ray class field resp.); let $p$ and $q$ be the first and second projections
(note that they are both surjective).
Our assumption translates into the statement that $p (q^{-1}(S)) = \{1\}$, i.e.
$q^{-1}(S) \subset 1 \times Cl\_{\mathfrak m}.$
Now choose $s \in S$, and suppose that $(g,1) \in G'$.  The previous paragraph together with the surjectivity of $q$ shows that also $(1,s) \in G'$.  Then $(g,1) (1,s) = (g,s) \in G',$
since $G'$ is a subgroup of the product.  But $(g,s)$ lies in $q^{-1}(S)$, hence $g = 1$.
In other words, if the second coordinate of an element of $G'$ is trivial, so is
the first.  Thus in fact $L'$ equals the ray class field of conductor $\mathfrak m,$
i.e. $L$ is contained in the latter field.  This is what had to be shown.
