Here's an answer for the special case when the base field is $\mathbb{Q}$. It involves a large bit of class field theory over $\mathbb{Q}$, so I'll be terse.
We start with the lemma which Buzzard mentioned.
Lemma - Let $K$, $L$ be finite Galois extensions of $\mathbb{Q}$. Then $K$ is contained in $L$ if and only if $\operatorname{sp}(L)$ is contained in $\operatorname{sp}(K)$ (with at most finitely many exceptions).
The proof of the lemma follows from the Chebotarev Density Theorem.
We now show that if the rational primes splitting in $K$ can be described by congruences, then $K/k$ is abelian.
Proof. Assume that the rational primes splitting in $K$ can be described by congruences modulo an integer $a$. This allows us to assume that $\operatorname{Sp}(K)$ contains the ray group $P_a$. The next step is to show that the rational primes lying in $P_a$ are precisely the primes of $\operatorname{sp}(\Phi_a(x))$. By the above lemma, this means that $K$ is contained in a cyclotomic field, hence is abelian.
