Here's an answer for the special case when the base field is Q. It involves a large bit of class field theory over Q, so I'll be terse.

We start with the lemma which Buzzard mentioned.

**Lemma** - Let K, L be finite extensions of Q. Then K is contained in L if and only if sp(L) is contained in sp(L) (with at most finitely many exceptions).

The proof of the lemma follows from the Chebotarev Density Theorem.

We now show that if the rational primes splitting in K can be described by congruences, then K/k is abelian.

**Proof**. Assume that the rational primes splitting in K can be described by congruences modulo an integer a. This allows us to assume that Sp(K) contains the ray group P_a. The next step is to show that the rational primes lying in P_a are precisely the primes of sp($\Phi_a(x)$). By the above lemma, this means that K is contained in a cyclotomic field, hence is abelian.