Here are three more elementary results whose most natural proof involves $p$-adic ideas. All three can be found in Cassels' book on Local Fields (edit mentioned by Laurent Berger in a comment).
The first one is Witt's proof of the theorem of Clausen and von Staudt in Chapter 1. It requires nothing more than the definition of the $p$-adic valuations; the idea is that $a\in{\bf Q}$ is in ${\bf Z}$ if $a\in{\bf Z}_p$ for every prime $p$.
The second one says that the order of every finite subgroup $G\subset{\mathrm GL}_n({\mathbf Q})$ divides
$$
\prod_l l^{\beta(l)}
$$
where $l$ runs over the primes and
$$\beta(l)=\lfloor n/(l-1)\rfloor+\lfloor n/l(l-1)\rfloor+\lfloor n/l^2(l-1)\rfloor+\cdots
$$
for $l\neq2$ and $\beta(2)=n+2\lfloor n/2\rfloor+\lfloor n/2^2\rfloor+\lfloor n/2^3\rfloor+\cdots$. See Theorem 2.1 in Chapter 4.
The third one is a theorem of Selberg which says that every finitely generated subgroup $G\subset{\mathrm GL}_n(k)$, where $k$ is a field of characteristic $0$, contains a normal torsionfree subgroup of finite index. See Theorem 4.1 in Chapter 5.
Note finally that the (Skolem)-Mahler-Lech theorem is Theorem 5.1 in the same chapter.
Addendum. Since I had already TeXed Witt's proof for my Notes, it is easy to reproduce it here :
Theorem (von Staudt--Clausen, 1840)
Let $k>0$ be an even integer, and let $l$ run through the primes. Then the
number
$$
W_k=B_k+\sum_{l-1\mid k}{1\over l} \tag 1
$$
is always an integer. For example, $\displaystyle
W_{12}=B_{12}+{1\over 2}+{1\over3}+{1\over5}+{1\over7}+{1\over 13}=1$.
[The British analyst Hardy says in his Twelve lectures
(p. 11) that this theorem was rediscovered by Ramanujan ``at a
time of his life when he had hardly formed any definite concept of
proof''.]
Proof (Witt) : The idea is to show that $W_k$ is a $p$-adic integer
for every prime $p$. More precisely, we show that $B_k+p^{-1}$ (resp. $B_k$)
is a $p$-adic integer if $p-1\mid k$ (resp. if not).
For an integer $n>0$, let $S_k(n)=0^k+1^k+2^k+\cdots+(n-1)^k$. Comparing the
coefficients on the two sides of
$$
1+e^T+e^{2T}+\cdots+e^{(n-1)T}={e^{nT}-1\over T}{T\over e^T-1},
$$
we get $\displaystyle S_k(n) =\sum_{m\in[0,k]}{k\choose m}{B_m\over k+1-m}n^{k+1-m}$. To recover $B_k$ from the $S_k(n)$, it is tempting to
take the limit $\displaystyle\lim_{n\to0}S_k(n)/n$, which doesn't make sense
in the archimedean world. If, however, we make $n$ run through the powers
$p^s$ of a fixed prime $p$, then, $p$-adically, $p^s\to0$ as $s\to+\infty$,
and
$$
\lim_{s\to+\infty}S_k(p^s)/p^s=B_k. \tag 2
$$
Let us compare $S_k(p^{s+1})/p^{s+1}$ with $S_k(p^s)/p^s$. Every
$j\in[0,p^{s+1}[$ can be uniquely written as $j=up^s+v$, where $u\in[0,p[$ and
$v\in[0,p^s[$. Now,
$$
\eqalign{
S_k(p^{s+1})
&=\sum_{j\in[0,p^{s+1}[}j^k=\sum_{u\in[0,p[}\sum_{v\in[0,p^s[}(up^s+v)^k\cr
% &\equiv p\left(\sum_{v\in[0,p^s[}v^k\right)
% +kp^s\left(\sum_{u\in[0,p[}u\sum_{v\in[0,p^s[}v^{k-1}\right)
% \pmod{p^{2s}}\cr
&\equiv p\left(\sum_v v^k\right)
+kp^s\left(\sum_u u\sum_v v^{k-1}\right)\pmod{p^{2s}}\cr
}$$
by the binomial theorem. As $\sum_{v}v^k=S_k(p^s)$ and
$2\sum_uu=p(p-1)\equiv0\pmod p$, we get
$$
S_k(p^{s+1})\equiv pS_k(p^s)\pmod{p^{s+1}},
$$
where, for $p=2$, the fact that $k$ is even has been used. Dividing
throughout by $p^{s+1}$, this can be expressed by saying that
$$
{S_k(p^{s+1})\over p^{s+1}}-{S_k(p^s)\over p^s} \in {\mathbf Z}_{(p)}
$$
is a $p$-adic integer, and therefore
$$
{S_k(p^r)\over p^r}-{S_k(p^s)\over p^s}\in Z_{(p)}
$$
for any two integers $r>0$, $s>0$, since ${\mathbf Z}_{(p)}$ is a subring of $\mathbf Q$.
Fixing $s=1$ and letting $r\to+\infty$, we see that $B_k-S_k(p)/p\in{\mathbf Z}_{(p)}$,
in view of $(2)$. We need a
Lemma.
$S_k(p)=\sum_{j\in[1,p[}j^k$ is $\equiv-1\pmod p$ if $p-1\mid k$ and $\equiv0\pmod p$ otherwise.
This is clear if $p-1\mid k$. Suppose not, and let $g$ be a generator of
$({\mathbf Z}/p{\mathbf Z})^\times$. We have $g^k-1 \not\equiv0$, whereas
$$
(g^k-1)\left(\sum_{j\in[1,p[}j^k\right)
\equiv (g^k-1)\left(\sum_{t\in[0,p-1[}g^{tk}\right)
\equiv g^{(p-1)k}-1\equiv0.
$$
It follows that $B_k+p^{-1}\in{\mathbf Z}_{(p)}$ if $p-1\mid k$ and $B_k\in{\mathbf Z}_{(p)}$
otherwise. In either case, the number $W_k$ (1), which can be written as
$$
W_k=\begin{cases}
(B_k+p^{-1})+\sum_{l\neq p}l^{-1} & \text{if }p-1\mid k\cr
(B_k)+\sum_l l^{-1} & \text{otherwise},\cr
\end{cases}$$
(where $l$ runs through the primes for which $l-1\mid k$) turns out to be a
$p$-adic integer for every prime $p$. Hence $W_k\in{\mathbf Z}$, as claimed.