I'd like to ask can we characterize the structure of finitely generated infinite p-group which has a unique subgroup of order p?
Can we say that these group are residually nilpotent? Any comments are welcome.
Here is a counterexample. Take $p>1001$, a prime. Take the free Burnside group $B(2,p)$ of exponent $p$ with $2$ generators $x,y$. It is infinite and has a central extension $\tilde B$ described in Aryan's book "Burnside problems and identities of groups" such that the intersection of any two cyclic subgroups of $\tilde B$ is infinite. The center $Z$ of $\tilde B$ is an infinite cyclic group. Let $H$ be the subgroup of index $p$ in $Z$. Then $Z/H$ is the unique subgroup of order $p$ in $\tilde B/H$, and every element of $\tilde B/H$ outside $Z/H$ has order $p^2$. By Zelmanov's theorem, since $\tilde B/H$ is infinite, it is not residually finite.
Another, more elementary, counterexample, of unbounded exponent can be found in Erschler, Anna, Not residually finite groups of intermediate growth, commensurability and non-geometricity. J. Algebra 272 (2004), no. 1, 154–172.