Added for Gowers: Maybe one reason why people fall into this error goes something like this: First you learn linear algebra, so you know about vector spaces, bases for same, splittings of same. Then you run into elementary abelian $p$-groups and recognize this as a special case of vector spaces. Then you learn the pleasant fact that all finite abelian $p$-groups are direct sums of cyclic $p$-groups, and a corresponding uniqueness statement. You notice that all of the cyclic subgroups of order $p^2$ in $\mathbb Z/p^2\times \mathbb Z/p$ are summands, and if you have a certain sort of inquiring mind then you also notice that not every subgroup of order $p$ is a summand: one of them is contained in a copy of $\mathbb Z/p^2$, in fact in all of those copies of it. Having learned so much, both positive and negative, from the example of $\mathbb Z/p^2\times \mathbb Z/p$, you may think that it shows all the interesting basic features of the general case and overlook the fact that in $\mathbb Z/p^3\times \mathbb Z/p$ there is a $\mathbb Z/p^2$ not contained in any $\mathbb Z/p^3$.
In any case, reputable people sometimes make this blunder; it happened to somebody here at MO just the other day.
