For binary forms a notation such as
$$
(a_0,a_1,\ldots,a_n)\!\!\!\!(x,y)^n
$$
means
$$
a_0 x^n+ a_1 \left(
\begin{array}{c} n \\\ 1\end{array}
\right)x^{n-1}y+
a_2 \left(
\begin{array}{c} n \\\ 2\end{array}
\right)x^{n-2}y^2+\cdots+a_n y^n
$$
If Cayley uses the notation with the pointy arrow on one of the parenthesis
he means the same thing without the binomial coefficients.
For $p$-ary forms, I believe there must be a choice of ordering of monomials
hopefully specified in the paper under consideration.