In line with Felipe Voloch's remark "Spec $\mathbb{Z}$ are where schemes really shine", I thought I'd also add this beautiful (expository and very readable!) paper of Serre:

"How to use finite fields for problems concerning infinite fields"

which makes crucial use of being able to do algebraic geometry over (finitely generated algebras over) $\mathbb{Z}$ in order to prove geometric statements (many of which are easily understable without any background in algebraic geometry!) over fields like $\mathbb{C}$ and $\mathbb{Q}$.