I just realized, that the arguments below work in greater generality:

Let $G$ be a $p$-group with an abelian subgroup $A$ of index $p$. Then there is a short exact sequence
$$1 \to Z(G) \to A \to G' \to 1.$$
In particular, $|Z(G)|\cdot |G'| = |A| = |G|/p$.

*Proof:* Let $g \in G$ such that $gA$ generates $G/A$. Then $G = \lt A,g \gt$. Since $A$ is abelian, the map
$$f: A \to G', a \mapsto [g,a]$$
is a homomorphism of groups. Let $B \le A$ be the kernel of $f$. The elements of $B$ commute with elements of $A$ and with $g$. Thus $B \le Z(G)$.

Next, let's show $Z(G) \le A$: Since $G$ is not abelian, $G'$ is non-trivial. As the intersection of the center of a $p$-group with a non-trivial normal subgroup is non-trivial, we find $a_0 \in Z(G) \cap G'$ of order $p$. Let's assume $f(A) = G'$ has already been shown. Then, there is $a_1 \in A$ such that $f(a_1) = a_0$, i.e. $ga_1g^{-1} = a_0a_1$. Now, if $z$ is any element of $Z(G)$, write $z=ag^i$ ($a \in A$). Hence
$$ 1 = [z,a_1] = ag^ia_1g^{-i}a^{-1}a_1^{-1} = (g^ia_1g^{-i})a_1^{-1} = ...= (a_0^ia_1)a_1^{-1} = a_0^i.$$
Since $a_0$ has order $p$, $i$ is a multiple of $p$ and from $g^p \in A$, $z \in A$ follows.

Thus, we can apply $f$ to $Z(G)$ and $f(Z(G))= \lbrace 1 \rbrace$ yields $Z(G) \le B$, whence $Z(G) = B$ and the sequence $1 \to Z(G) \to A \xrightarrow[]{f} G' \to 1$ is exact.

Finally, let's show $f(A) = G'$. From definition, $[g,a] \in f(A)$ for all $a \in A$. Suppose $[g^i,a] \in f(A)$ for all $a \in A$. Then,
$$[g^{i+1},a] = g(g^iag^{-i})g^{-1}a^{-1} = g[g^i,a]ag^{-1}a^{-1}=g[g^i,a]g^{-1}(gag^{-1}a^{-1})$$
$$=[g^i,gag^{-1}][g,a] \in f(A).\hspace{125pt}$$
If $x=ag^i, y=bg^j \in G$, then a simple computation, using that $A$ is abelian, shows $[x,y] = [g^i,b][g^j,a^{-1}] \in f(A)$, finishing the proof. qed.

*Old version:*

$|G'| = p$ is a strong condition that severely limits the possibilities for $G$. In fact, it can be shown:

If $G$ is a non-abelian $p$-group with $|G'| = p$, which has an abelian subgroup $A$ of index $p$, then $Z(G)$ is an index-$p$ subgroup of $A$. In particular, $|Z(G)| = |G| / p^2$.

*Remark 1:* $G$ can be of any order $p^n$

*Remark 2:* From the proof below, it should not be to hard, to classify all groups $G$ from the statement.

*Proof:* Since $G/A$ is abelian, $G' \le A$. Let $G' = \lt a_0 \gt$. Since the center of a $p$-group non-trivially intersects every non-trivial normal subgroup, $a_0 \in Z(G)$ follows. Let $g \in G$ represent a generator of $G/A$ (note: $G = \lt A,g \gt$) and let
$$f: A \to G', a \mapsto [g,a].$$
Since $A$ is abelian and $G$ non-abelian and $G'$ cyclic of prime order, $f$ is a surjective homomorphism. Let $B$ be the kernel of $f$ and let $a_1 \in A$ with $[g,a_1] = a_0$, i.e. $ga_1g^{-1}=a_0a_1$. Because elements from $B \le A$ commute with $g$ and $A$, one conlcudes $B \le Z(G)$.

Next I show $Z(G) \le A$: Let $z=ag^i \in Z(G)$. Thus
$$1 = [z,a_1] = ag^ia_1g^{-i}a^{-1}a_1^{-1}= (g^ia_1g^{-i})a_1^{-1}=...=(a_0^ia_1)a_1^{-1}=a_0^i$$
and since $a_0$ has order $p$, $i$ is a multiple of $p$, implying $z \in A$ (note: $g^p \in A$).

Hence $B \le Z(G) \lvertneqq A$. Finally, surjectivity of $f$ implies $(A:B) = p$, whence $Z(G) =B$ and $Z(G) = |G|/p^2.\quad$qed.