If $|G|=p^n,n>1$, and $1=G_n \subseteq G_1\subseteq \cdots \subseteq G_0=G$, is a composition series, with cyclic-$p$ quotients, and $P$ is set of all automorphisms of $G$, such that for every $u\in P$, $u(x)x^{-1}\in G_{i+1}$ $\forall x\in G_i$, then $P$ is a $p$ subgroup of $Aut(G)$.
What about converse: given $p$ subgroup $P$ of $Aut(G)$,(where $|G|=p^n, n>1$) does there exist a composition series $1=G_n \subseteq G_1\subseteq \cdots \subseteq G_0=G$ of $G$ with cyclic p-quotients such that for $u\in P$, $u(x)x^{-1}\in G_{i+1}$ $\forall x\in G_i$? Can $G_i$ chosen to be normal in $G$?
(This will be useful in finding some explicit automorphisms of groups of order $p^3$).