The probability distribution of $U^p$ for $U$ uniformly distributed with the Haar measure in $\text{U}(n)$ has been calculated by Eric Rains in Images of eigenvalue distributions under power maps.
For $p=2$ and $n$ even the eigenvalue distribution of $U^2$ is obtained by taking the union of the eigenvalues of two independent matrices $U_1$ and $U_2$, uniformly distributed in $\text{U}(n/2)$. For $n$ odd the two independent matrices are taken from $\text{U}((n+1)/2)$ and $\text{U}((n-1)/2)$.
So $U^2$ is only Haar random in the trivial case $n=1$, but not for $n>1$. For $n=2$, in particular, the two eigenvalues of $U^2$ are independent, in contrast to the two eigenvalues of $U$.
To see how squaring the $2\times 2$ unitary $U$ removes the correlations, use that the joint probability distribution of the two eigenvalues $\lambda_1$ and $\lambda_2$ of $U$ is
$$P(\lambda_1,\lambda_2)\propto|\lambda_1-\lambda_2|^2=2-2\,{\rm Re}\,\lambda_1\bar{\lambda}_2.$$
Squaring $U$ identifies $\pm\lambda_i$, so the contributions from the correlator ${\rm Re}\,\lambda_1\bar{\lambda}_2$ cancel and $P(\lambda_1^2,\lambda_2^2)$ becomes independent of $\lambda_i$.