Hall algebra for non-abelian $p$-groups? According to WP article on Hall algebras one counts the number of abelian subgroups in an abelian group with fixed type of subgroup, group, quotient.
Two things are claimed:

*

*These numbers are  polynomials in $p$.


*Using these numbers one naturally defines the algebra structure on the
isomorphism classes of abelian groups which appears to be associative and commutative.
Question: What happens if we consider all $p$-groups, not just abelian one ? Will same/similar claims be true ?
PS: The natural context for the question seems to me some categories with finite number of exact triples $A\to B\to C$ for fixed $A,B,C$. So a natural generalization is: what the properties of the categories for which
we can define associative algebra? commutative algebra?
 A: Well, the first claim would be impossible to properly describe in the way you want, because unlike abelian $p$-groups which are always indexed by partitions, non-abelian $p$-groups don't have the same classification for different $p$: you don't even necessarily have the same number of groups of order $p^n$ for fixed $n$ and varying $p$!
Regarding the second claim, for any given $p$ the algebra generated should still be associative, as one can still count filtrations with the desired quotients, but commutativity will be lost; for instance, given $p=2$, the quaternion group $Q_8$ can be generated from a short exact sequence
$$1 \rightarrow \mathbf{Z}/2\mathbf{Z} \rightarrow Q_8 \rightarrow \mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/2\mathbf{Z} \rightarrow 1 $$
but there exists no injective map $\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/2\mathbf{Z} \rightarrow Q_8 $, so the products of the elements corresponding to those two groups do not equal each other.
The proper generalization is instead to consider abelian categories where $\hom(M,N)$ and $\mathrm{Ext}^1(M,N)$ are finite.  Aside from the classical case of abelian $p$-groups, I believe the best studied is that of quiver representations over a finite field $\mathbf{F}_q$; these, too, form polynomials over $q$, though are non-commutative in all but trivial cases.
A: Just ran into this old question. As other answers have noted (1) for fixed $n$, the number of groups of order $p^n$ depends on $p$ in a complicated way and (2) if you try to make an algebra with $p$-groups as a basis and multiplication by copying the Hall formula, it won't be commutative. I am writing to note that it won't be associative either!
In the Hall algebra, the coefficient of $[X]$ in $[A][B]$ is the number of submodules $Y$ of $X$ such that $Y \cong A$ and $X/Y \cong B$. If we make the analogous definition for groups, we would want the number of normal subgroups $Y \trianglelefteq X$ such that $Y \cong A$ and $X/Y \cong B$
So the coefficients of $[X]$ in ${\big(} [A][B] {\big)} [C]$ and in $[A] {\big(} [B][C] {\big)}$ are, respectively,
(1) The number of chains $Z \trianglelefteq Y \trianglelefteq X$ with $Z \cong A$, $Y/Z \cong B$ and $X/Y \cong C$ and
(2) The number of pairs $(Z, Y')$ where $Z \trianglelefteq X$ and $Y' \trianglelefteq X/Z$, with $Z \cong A$, $Y' \cong B$ and $(X/Z)/Y' \cong C$.
Replacing $Y'$ by its preimage $Y$ in $X$, (2) is the same as
(2') The number of chains $Z \trianglelefteq Y \trianglelefteq X$ which have $Z \trianglelefteq X$, with $Z \cong A$, $Y/Z \cong B$ and $X/Y \cong C$.
In other words, the difference between (1) and (2') is that, in (1), we require that $Z \subseteq Y \subseteq X$ is a subnormal series, and in (2') we require that it is a normal series.
Therefore, any $p$-group with a subnormal series that isn't a normal series will give a counterexample. For example, the dihedral group of order $8$ has $7$ subnormal series with quotients $(C_2, C_2, C_2)$, but only $3$ of them are normal series.
A: This sort of goes against the definition of a Hall Algebra... The Hall Polynomials for which the algebra is defined on have the condition that they are abelian... So there wouldn't be any hall algebras for nonabelian p-groups. 
