The group $F_4^{-20}$ is according to Yokota the automorphism group of the real Jordan algebra $J(1,2,\mathbb{O}) = \{X\in \mathrm{M}(3,\mathbb{O}\otimes_\mathbb{R}\mathbb{C}\\, |\\, I_1 \overline{X}^tI_1 = X \}$ where $I_1 = \mathrm{diag}(-1,1,1)$. Since the invariant form is given by $A\mapsto \mathrm{Tr}(A^2)$ restricted to the space of trace-free matrices the result follows quite easily.
The other two cases $F_4^{-52}$, $F_4^4$ follow similarly since they are the automorphism groups of $J(3,\mathbb{O}) = \{ X\in M(3,\mathbb{O})\\,|\\, X^t =X \}$ and $J(3,\mathbb{O}) = \{ X\in M(3,\mathbb{O}')\\,|\\, X^t=X \}$ respectively.