There is an isomorphism between (rational) correspondences on a curve $C/\mathbb{F}_p$ orthogonal to the "valence zero" ones (i.e. orthogonal under intersection pairing to $\{*\}\times C$ and $C\times \{*\}$) and endomorphisms of the Jacobian $J(C)$, giving a pretty and applicable demonstration of how $J(C)$ "is" the motivic weight-$1$ part of the curve, and illustrating the simplest case of the Lefschetz trace formula. One can show that this is an isomorphism in a pretty nice geometric way (from an endomorphism, get a correspondence by looking at the group chunk $C\times C^{(g)}\subset J$; from a correspondence with dim-zero fibers which sends each point to a principal divisor, assemble the collection of rational functions into a global rational function showing the correspondence is principal as well).
Further, transpose becomes Rosati involution, and intersection number becomes the negative of trace pairing. The former can be demonstrated using the functoriality of the Weil pairing, which can naturally be proven geometrically without recourse to cohomology, e.g. in section 23 of Mumford's book. The latter, however, I am struggling with.
This is all immediate once we have a Weil cohomology theory (and in fact provides a pretty good motivation for Weil cohomology, which is more or less the historical one), but I would really like to see how to do it without invoking cohomology. This is equivalent to showing that the numerator of the zeta function is the characteristic polynomial of Frobenius on the Jacobian (defined via degree, or via an $l$-adic Tate module; one can prove without cohomology that the latter definitions are independent of $l$ and agree with the former, e.g. in Milne's notes). I don't think any of the "computational" approaches to proving the Weil conjectures for curves can yield this. Another equivalent formulation is the trace formula for fixed points; not sure if either of these offer a better approach.
So my question is, is there a nice way (ideally a very geometric one) to demonstrate this correspondence without appeal to cohomology? I have a vague impression that this result was more-or-less known to Weil without this machinery, but as I mentioned I didn't find that any of his proofs of his conjectures actually established this link.
It seems very difficult to link the geometry of the Jacobian to intersection numbers - the trace pairing in particular. I saw a comment which mentioned Mumford's book has this $l$-adic argument, but I combed it and it seems to have only the argument establishing the Weil conjectures for abelian varieties, which goes pretty smoothly since one can simply look at the kernel of the endomorphism $\text{Fr}^n-1$.
The "numerical" proofs of the Weil conjectures seem to buy is a bunch of isolated things like the fact that both are positive pairings, that both are preserved by the Rosati involution, that the numerator of the zeta function is an integral polynomial with roots of the correct norm, that its evaluation at $1$ is the number of rational points of the Jacobian (from the Tate's thesis approach), that the pairing of the diagonal/identity with itself is $2g$. I don't know if this collection of facts is enough to determine the pairing. The most promising is the interpretation of the Weil form as a trace, which is used to calculate the trace pairing to prove positivity (e.g. in sections 20-21 of Mumford's book); this yields $$\text{Tr}(l_A \circ l_{A^T}) = 2g \frac{(\Theta^{g-1} \cdot l^*_A(\Theta))}{(\Theta^g)}$$ where $A$ is our correspondence with associated endomorphism $l_A$. But I still can't figure out how to show that this is $-A\cdot A$.
