Edit 2: For those who understandably don't want to read such a long post, I think Voloch's suggestion reduces the problem to asking whether $j$-invariants of supersingular curves are 3rd powers in $\mathbb{F}_{p^2}$. I plugged in a few small values and they were third powers, so this claim seems plausible. Is there is a reference?

I think it might be possible to show that, if $j$ is a supersingular $j$-invariant of an elliptic curve over $\overline{\mathbb{F}_{p}}$ ($p>3$), then the polynomial $(256x^{12}+1)^3 - j x^{12}$ splits over $\mathbb{F}_{p^2}$.

For example, if $p=137$, then 22 is a supersingular $j$-invariant and the polynomial $(256y^{12}+1)^3 - 22 y^{12}$ factors as \begin{align*} 59(y + 67) (y - 13) (y + 13) (y - 67) (y^2 + 13y + 32) (y^2 + 21y - 16) (y^2 + 28y - 16) (y^2 + 45y + 16) (y^2 + 60y + 16) (y^2 + 67y - 32) (y^2 - 67y - 32) (y^2 - 60y + 16) (y^2 - 45y + 16) (y^2 - 28y - 16) (y^2 - 21y - 16) (y^2 - 15y + 16) (y^2 - 13y + 32) (y^2 - 7y - 16) (y^2 + 7y - 16) (y^2 + 15y + 16) \end{align*}

over $\mathbb{F}_{137}$. This works for all supersingular $j$-invariants in $\mathbb{F}_{p}$ that I tested so far, but I haven't tested one in $\mathbb{F}_{p^2}\backslash\mathbb{F}_{p}$ yet.

Does anybody know if something like this exists in the literature, or if this in fact easy to show?

Edit: If we write $j=2^8 \frac{(\lambda^2-\lambda+1)^3}{\lambda^2(\lambda-1)^2}$, where the $\lambda$ comes from Legendre form, then the polynomial $(256x^{12}+1)^3 - j x^{12}$ splitting over $\mathbb{F}_{p^2}$ is the same as \begin{align*} \lambda^2(\lambda-1)^2 (256x^{12}+1)^3 - 2^8(\lambda^2-\lambda+1)^3x^{12}=(256\lambda x^{12} - \lambda^2 + 2\lambda - 1) (256(\lambda - 1) x^{12} + \lambda^2) (256(\lambda^2 - \lambda )x^{12} - 1) \end{align*} splitting.

So from above, the polynomial splits when $\frac{(1-\lambda)^2}{256\lambda},\frac{\lambda^2}{256(1-\lambda)},-\frac{1}{256\lambda(1-\lambda)}$ are all 12th powers. But, since the appendix cited by Felipe Voloch says $\lambda$ is a 4th power for supersingular curves, combined with the fact that $1-\lambda$ gives the same $j$-invariant, if any of the three values above is a 12th power, all of them are. Therefore, this reduces to showing that, if $\lambda$ corresponds to a supersingular elliptic curve in Legendre form, then $\frac{\lambda^2(1-\lambda)^2}{256}$ is a 3rd power in $\mathbb{F}_{p^2}$.

But from the formula for $j$, we see that $\frac{\lambda^2(1-\lambda)^2}{256}=\frac{(\lambda^2-\lambda+1)^3}{j}$, this reduces to showing that $j$ is a 3rd power (so at this point, either this fact is standard or a mistake renders a contradiction).