Lots of the answers are going to be of the form "read book X" so I'll start: Serre's Course in Arithmetic is the best conceivable answer to the question "where could I find a concise, clearly written compilation of results in arithmetic geometry that can be proved without advanced machinery?" It's really a beautiful book.

The kind words about my article are much appreciated.