MathOverflow will be down for maintenance for approximately 3 hours, starting Monday evening (06/24/2013) at approximately 9:00 PM Eastern time (UTC-4).

Theorem: For every prime $p$, there is a finite non-solvable Galois extension of $\mathbf{Q}$ ramified only at $p$.
For $p\geq 11$ this is not so hard, but for smaller primes the only known examples come via the Galois representations associated with modular forms. See e.g. this paper on the case $p=2$, which uses Hilbert modular forms!