How to make Burnside's formula compatible with point counting for varieties over finite fields?

Question

If $G$ is a finite group acting on a finite set $X$, we have Burnside's formula that counts the number of orbits $|X/G|$ as: $$ |X/G| = \frac1{|G|} \sum_{g\in G} |X^g|, $$ with $X^g$ being the set of elements in $X$ fixed by $g$.

Now, consider the finite set $X$ of $\mathbb F_p$-points of $\mathbb P^1_{\mathbb F_p}$, the one dimensional projective space over the finite field $\mathbb F_p$, with $p$ elements, and the finite set $Y:=X\times X$ (Cartesian product). The 2-element group $G:=\{e,g\}$ acts by permuting the factors on $Y$ ($G$ is the symmetric group on 2 elements, and $e$ its identity), and it is well-known that (it follows, for example, from the Zeta function computation in https://math.stackexchange.com/q/799101): $$ Y/G = Sym^2(\mathbb P^1_{\mathbb F_p}) = \mathbb P^2_{\mathbb F_p}. $$

As such, we have $|Y/G|=|\mathbb P^2_{\mathbb F_p}|=p^2+p+1$.

On the other hand, one would say that $|Y^e|=|Y|=|X|^2=(p+1)^2$, and $|Y^g|=|X|=p+1$, since the elements fixed under the permutation are in the diagonal of $Y=X\times X$, isomorphic to $X$, so Burnside's formula gives, apparently, the wrong answer: $$ |Y/G|=\frac12((p+1)^2+(p+1))=\frac{p^2+3p+2}{2}.$$

Maybe I am using the wrong notion of $\mathbb F_p$-points of a variety (/scheme?), but I believe there should be a simple explanation of why the computations do not match. Any help is appreciated.

Answer 1

The issue is that $X(\mathbb{F}_p)/G$ is not the same thing as $(X/G)(\mathbb{F}_p)$. A simpler example is to take $p$ odd, $X = \mathbb{A}^1$ and let $S_2$ act by $\pm 1$. There are $\tfrac{p+1}{2}$ orbits, but the quotient space is $\mathbb{A}^1$ with the quotient map $x \mapsto x^2$. The quotient space has $p$ $\mathbb{F}_p$-points, but only $\tfrac{p+1}{2}$ of them (the squares) are in the image of $X(\mathbb{F}_p)$.

Points of $(X/G)(\mathbb{F}_p)$ index Frobenius stable $G$-orbits, so the other $\tfrac{p-1}{2}$ points correspond to the orbits of the form $\{ \pm x \}$ where $x^p = -x$ (other than the point $x=0$).

If you want to count $(X/G)(\mathbb{F}_p)$, there is a combined Burnside/Lefschetz formula. Choose $\ell$ relatively prime to $p$ and $|G|$, then a formula of Grothendieck tells us that $H^j(X/G, \mathbb{Q}_{\ell}) \cong H^j(X, \mathbb{Q}_{\ell})^G$, and this isomorphism is Frobenius equivariant. So the trace of Frobenius on $H^j(X/G, \mathbb{Q}_{\ell})$ is the same as the trace of Frobenius restricted to the subspace $H^j(X, \mathbb{Q}_{\ell})^G$. Now, the linear operator $\tfrac{1}{|G|} \sum_{g \in G} g$ on $H^j(X, \mathbb{Q}_{\ell})$ is an idempotent whose image is $H^j(X, \mathbb{Q}_{\ell})^G$. So the trace of Frobenius restricted to $H^j(X, \mathbb{Q}_{\ell})^G$ is the same as the trace of $\tfrac{1}{|G|} \sum_{g \in G} \text{Frob} \circ g$. So we get $$\#((X/G)(\mathbb{F}_p)) = \tfrac{1}{|G|} \sum_{g \in G} \sum_j (-1)^j \text{Tr}{\big(}\text{Frob} \circ g : H^j(X, \mathbb{Q}_{\ell}) \longrightarrow H^j(X, \mathbb{Q}_{\ell}){\big)}.$$

I am not sure that there is a Burnside/Lefschetz style formula for $\#X(\mathbb{F}_p)/G$. Here is a troubling example: Take $p$ odd, $X = \mathbb{P}^1$ and let $S_2$ act by $[x:y] \mapsto [cy : x]$. Then $X(\mathbb{F}_p)$ has $p+1$ points. If $c$ is a quadratic residue, then there are two fixed points for $S_2$, namely $[\pm \sqrt{c}:1]$, otherwise there are none. So there are either $\tfrac{p+1}{2}$ or $\tfrac{p-1}{2}$ orbits for $S_2$ on $X(\mathbb{F}_p)$ depending on whether or not $c$ is a quadratic residue. I find it hard to imagine a Burnside/Lefschetz style formula which could take this information into account.

---

It might help to say that there is a purely topological version of this question. Let $G$ be a finite group and let $X$ be a compact topological space with an action of $G$ and also with an endomorphism $\phi$ that commutes with the $G$-action. (More generally, we could imagine that there is an automorphism $\sigma$ of $G$ with $\phi \circ g = \sigma(g) \circ \phi$.)

Then the two questions are how to count $(X/G)^{\phi}$ and how to count $(X^{\phi})/G$. These aren't the same thing: If there is a $G$-orbit which $\phi$ permutes nontrivially, then it will contribute to the first count but not the second.