The issue is that $X(\mathbb{F}_p)/G$ is not the same thing as $(X/G)(\mathbb{F}_p)$. A simpler example is to take $X = \mathbb{A}^1$ and let $S_2$ act by $\pm 1$. There are $\tfrac{p+1}{2}$ orbits, but the quotient space is $\mathbb{A}^1$ with the quotient map $x \mapsto x^2$. The quotient space has $p$ $\mathbb{F}_p$ points, but only $\tfrac{p+1}{2}$ of them (the squares) are in the image of $X(\mathbb{F}_p)$.
Points of $(X/G)(\mathbb{F}_p)$ index Frobenius stable $G$-orbits, so the other $\tfrac{p-1}{2}$ points correspond to the orbits of the form $\{ \pm x \}$ where $x^p = -x$ (other than the point $x=0$).
If you want to count $(X/G)(\mathbb{F}_p)$, there is a combined Burnside/Lefschetz formula. Choose $\ell$ relatively prime to $p$ and $|G|$, then a formula of Grothendieck tells us that $H^j(X/G, \mathbb{Q}_{\ell}) \cong H^j(X, \mathbb{Q}_{\ell})^G$, and this isomorphism is Frobenius equivariant. So the trace of Frobenius on $H^j(X/G, \mathbb{Q}_{\ell})$ is the same as the trace of Frobenius restricted to the subspace $H^j(X, \mathbb{Q}_{\ell})^G$. Now, the linear operator $\tfrac{1}{|G|} \sum_{g \in G} g$ on $H^j(X, \mathbb{Q}_{\ell})$ is an idempotent whose image is $H^j(X, \mathbb{Q}_{\ell})^G$. So the trace of Frobenius restricted to $H^j(X, \mathbb{Q}_{\ell})^G$ is the same as the trace of $\tfrac{1}{|G|} \sum_{g \in G} \text{Frob} \circ g$. So we get $$\#((X/G)(\mathbb{F}_p)) = \tfrac{1}{|G|} \sum_{g \in G} \sum_j (-1)^j \text{Tr}{\big(}\text{Frob} \circ g : H^j(X, \mathbb{Q}_{\ell}) \longrightarrow H^j(X, \mathbb{Q}_{\ell}){\big)}.$$
I think there should also be a Burnside/Lefschetz style formula for $\#X(\mathbb{F}_p)/G$, but I haven't seen it and wasn't able to work it out quickly.