Yes. It's necessary and sufficient to show that every finite abelian group admits a nondegenerate quadratic form valued in a finite cyclic group. The following slightly stronger statement is true: every finite abelian $p$-group admits a nondegenerate quadratic form valued in $C_{p^k}$ for some $k$ (this suffices by the Chinese remainder theorem). So let's prove this.

If $A$ is a finite abelian $p$-group for $p$ an odd prime, we can pick any isomorphism $A \cong A^{\ast}$ from $A$ to its Pontryagin dual $A^{\ast} = \text{Hom}(A, \mathbb{Q}/\mathbb{Z}) \cong \text{Hom}(A, \mathbb{Q}_p/\mathbb{Z}_p)$ and we'll get a nondegenerate bilinear form $B : A \times A \to C_{p^k}$ whose associated quadratic form $Q : A \to C_{p^k}$ is nondegenerate. As you say in the comments, this almost but doesn't quite work when $p = 2$.

When $p = 2$ the following slight modification works. Again pick an isomorphism $A \cong A^{\ast}$ to the Pontryagin dual and get a nondegenerate bilinear form $B : A \times A \to C_{2^k}$. Now we do something a bit funny. Consider the inclusion (not a group homomorphism!) $C_{2^k} \to C_{2^{k+1}}$ given by $k \mapsto k$, thinking of elements of $C_n$ as elements of $\mathbb{Z}/n$. Composing this inclusion with $B$ gives a map (not a bilinear map!) $B' : A \times A \to C_{2^{k+1}}$. Now I claim that the diagonal $Q(a) = B'(a, a)$ of this map is a nondegenerate quadratic form. We clearly have $Q(-a) = B'(-a, -a) = B'(a, a)$, and the associated bilinear form $Q(a + b) - Q(a) - Q(b)$ recovers $B$, now taking values in $2 C_{2^{k+1}} \cong C_{2^k}$.

In particular, when $A = C_2$ we get the quadratic form $Q : C_2 \to C_4$ given by $Q(0) = 0, Q(1) = 1$. This quadratic form can be interpreted as a cohomology class in $H^4(B^2 C_2, C_4)$ and so in turn a cohomology operation $H^2(-, C_2) \to H^4(-, C_4)$ which I believe is exactly the Pontryagin square.

4more comments