Let $\Gamma$ be the kernel of the reduction map $GL_n(\mathbb{Z}/p^3\mathbb{Z}) \to GL_n(\mathbb{Z}/p\mathbb{Z})$. $\Gamma$ is a non-abelian group of order $p^{2n^2}$ and $p$-rank $n^2$. Its mod-$p$ cohomology is
$$ H^\ast(\Gamma;\mathbb{F}_p) = \mathbb{F}_p[x_1,...,x_{n^2}] \otimes \Lambda(y_1,...,y_{n^2}).$$
In particular the cohomology ring equals those of the abelian group $(\mathbb{Z}/p^2\mathbb{Z})^{n^2}.$ As a reference for the cohomology of $\Gamma$ see this paper of Browder and Pakianathan (Theorem 1 and the example after Prop. 2.5).
Remark 1: There is a classification of finite groups those mod-p cohomology ring equals the cohomology of an abelian p-group. This can be used to obtain further examples. As reference see Theorem B in the following paper of Weigel.
Remark 2: In the examples above only the cohomology rings as abstract rings agree. I can remember of having seen somewhere an example of non-isomorphic 2-groups of the same order those cohomology rings even agreed as Steenrod algebras.