The sum $\sum_\chi \chi(\alpha)^2$ is the size of the centralizer $z_\mu=\frac{n!}{|K_\mu|}=1^{m_1}m_1! 2^{m_2}m_2!\cdots n^{m_n} m_n!$ if $\alpha$ has cycle type $\mu=1^{m_1}2^{m_2}\ldots$, so
$$\frac{1}{n!} \sum_{\alpha\in \mathfrak S_n} \left(\sum_\chi \chi(\alpha)^2\right)^2=
\frac{1}{n!} \sum_{\mu\vdash n} |K_\mu| z_\mu^2=\sum_{\mu\vdash n} z_\mu
=\sum_{\mu\vdash n}\sum_{\lambda\vdash n} (\chi^{\lambda}_\mu)^2,$$
as desired.

Although this identity is clear, there is something interesting here when you stand back. When you first come to representations, you'd maybe first think of $\mathfrak S_n$ acting on itself by left multiplication, which you quickly check affords the character $\rho$ which gives $|\mathfrak S_n|$ at the identity element and $0$ elsewhere. So, using the inner product, you immediately decompose this representation into irreducibles and find
$$\rho=\sum_\chi \chi(1)\chi,$$
i.e. each irreducible shows up with multiplicity its dimension.
Now you might try $\mathfrak S_n$ acting on itself by conjugation, which affords the character $\psi$ with value $z_\mu$ at any element of cycle type $\mu$. How does this one decompose? As we saw before,
$$\langle \psi,\chi^\lambda\rangle=\sum_{\mu\vdash n}\chi^\lambda_\mu.$$
But using that
$$\sum_{\lambda\vdash n} (\chi^\lambda_\mu)^2=z_\mu,$$
we also have the expression

$$\psi=\sum_{\chi} \chi^2,$$
since evaluating the RHS at an element with cycle type $\mu$ indeed gives $z_\mu$.

I should mention that the individual products $\chi^2$ are not generally well understood. For example, there is a conjecture that if $\lambda$ is a staircase shape $(n,n-1,n-2,\ldots,1)$, then the square of $\chi^\lambda$ contains at least one copy of each irreducible, i.e.
$$\langle (\chi^\lambda)^2,\chi^\nu\rangle\geq 1$$
for all $\nu\vdash \binom{n+1}{2}$.
This is a folklore conjecture due to Jan Saxl. (Of course, if one knew how to decompose arbitrary products $\chi\phi$, then one would know how to decompose the squares $\chi^2$, and if one knew how to decompose squares, then one would know how to decompose the conjugation representation.)