Minhyong's comments indicate the issue here. If I want to come up with an extension unramified outside $p$ then why not look at the 2-dimensional mod $p$ representation attached to the $\Delta$ function? This works for all but a very small set of $p$, where either the mod $p$ representation is degenerate, or $p$ is so small that $GL(2,p)$ is solvable anyway. For example the semisimple mod 2 representation attached to the $\Delta$ function is trivial. Gross' question was how to deal with this small set of primes.
For these small primes one can try other level 1 forms, of course. For example $p=691$ is a funny case where the representation attached to Delta is reducible, but for $p=691$ you can just use the level 1 weight 16 form instead. The problem with the smaller primes is harder to deal with, because e.g. a modular mod 2 representation unramified outside 2 must be reducible by an old theorem of Tate (look at bounds on discriminants -- the argument is delicated). So one has to try and look elsewhere. The trick with $p=2$ is due to Lassina Dembele and you can get the paper at his website
http://www.warwick.ac.uk/staff/L.Dembele/
Classical modular forms don't cut the mustard, so one seeks to try the same trick with Hilbert modular forms defined over a totally real field ramified only at 2. Such totally real fields are not hard to find, so the issue is now the following computational one -- how to compute the level 1 forms? Dembele did this, and found an explicit example which gave a Galois representation into $GL(2,k)$ with $k$ of size 8 if I remember correctly (I have to get the kids out of bed -1 minutes ago so can't check any more details).