I answer here positively the second question (it's completely independent of the first one so it could have been 2 distinct posts).

Let $p\le n$ be prime. Let $C\subset S_n$ be the group generated by the cycle $(1\dots p)$. Let $C$ act on $S_n$ by conjugation. It preserves the set $Q$ of squares. Let $Q^C$ be the $C$-fixed points (that is, the set of squares centralizing $C$). The $C$-action on $Q\smallsetminus Q^C$ is free, hence $Q\smallsetminus Q^C$ has cardinality divisible by $p$. The centralizer of $C$ is $C\times S_{n-p}$ (where $S_{n-p}$ is the pointwise stabilizer of $\{1,\dots,p\}$), so if $p$ is odd its number of squares is $\#(Q^C)=p\mathrm{sq}(n-p)$. So $Q$ has cardinality divisible by $p$.

If $p=2$ the previous argument fails since then $\#(Q^C)=\mathrm{sq}(n-2)$ (instead of $2\mathrm{sq}(n-2)$). So we assume $n\ge 4$ and now let $C$ be generated by the 4-cycle $(1234)$. Then $Q\smallsetminus Q^C$ has even cardinality, and the centralizer of $C$ is $C^2\times S_{n-4}$, and has $\#(Q^C)=2\mathrm{sq}(n-4)$ squares (here $C^2$ means the set of squares in $C$, which is a subgroup of order 2). Thus $Q$ has even cardinality.