Been going through Alan Baker's A Comprehensive Course in Number Theory. Very interesting book, although the way proofs are presented sometimes throws me off a little.

I usually read through a chapter multiple times and then try and solve a few exercises (I can't say I'm proficient or smart enough to solve all of them). I wish the answers to some of the exercises were somewhere.

In any case, here's one that I've been struggling with, and I've found no simple proof (I think one is meant to only use the tools given in the chapter, which are Euler's criterion, Gauss' Lemma and the quadratic reciprocity law, both for Lagrange and for Jacobi symbols, or in previous chapters, such as generators):

Let $p$ be an odd prime and $a$ be an integer not divisible by $p$. Prove that, if $a$ is a quadratic residue $\bmod p$, then it is a quadratic residue $\bmod p^k$ for all positive integers $k$.