As a first-round answer and some detailed about this problem:

Let $G$ be a $p-$group of odd order such that every abelian normal subgroup has at most $k$ generators, then every subgroup of $G$ has at most $C(k+1,2)$ generators.  
For your question, if we have $p>2$ and $k=1$, it is a classical result that $G$ is cyclic; see the thesis which I introduced below.

In the thesis "Abelian subgroups of $p-$groups"[ The thesis][1] by Soo-Seng Siah, this problem and its generalization is studied with related to the term of "depth" and "normal-depth" of a $p-$group $G$. You can see the section 4 of this thesis for some more information. 


  [1]: http://summit.sfu.ca/system/files/iritems1/4186/b13538421.pdf