What is the correct notion of representation for abelian varieties?

Question

Zeroth question - am I right that in the "ordinary" sense an abelian variety does not possess any representations at all?

More precisely, a representation of an algebraic group $G$ (over an algebraically closed field $K$, say, over complex numbers) is a homomorphism $G\to\operatorname{GL}(V)$ for some $K$-vector space $V$, which is a morphism of algebraic varieties. I have vague feeling that if $G$ is a projective variety, hence an abelian variety, then there are no nontrivial such homomorphisms.

If the answer to this zeroth question is negative, then my actual ("number one") question would be whether there is a classification of such representations.

(As @Wojowu explains in a comment below, this is indeed true)

If it is positive, then the question is whether there exists a modification of the notion of representation that would give some meaningful result - mostly, would allow studying an abelian variety $G$ through such representations.

Possible approaches would, maybe, include allowing "representations with singularities", say, instead of polynomial homomorphisms to allow rational homomorphisms $G\to\operatorname{GL}(V)$. Or, say, one might consider $G$-equivariant vector bundles over $G$ (whatever this means). Or, one might look at algebraic homomorphisms $G\to\operatorname{Aut}(A)$ where $A$ is some commutative "thing" in $K$-varieties such that the algebraic group $\operatorname{Aut}(A)$ admits nontrivial algebraic homomorphisms from abelian varieties to it. Subquestion: are there such $A$? Can it be, say, another abelian variety?

Maybe one more hopefully simpler subquestion. Let $\operatorname{Aut}(|G|)$ be the algebraic group of all algebraic automorphisms of the underlying algebraic variety $|G|$ of $G$. Then (presumably) the map assigning to $x\in G$ the multiplication-by-$x$ operator $G\to G$ is an injective algebraic homomorphism $G\to\operatorname{Aut}(|G|)$, so $\operatorname{Aut}(|G|)$ contains a copy of $G$ as a subgroup. What are, if any, subgroups in between? How does $\operatorname{Aut}(G)$ sit inside $\operatorname{Aut}(|G|)$? Is this $\operatorname{Aut}(|G|)$ studied somewhere?

Answer 1

I am also very far from an expert here, but I think there's a case to be made that the "correct notion" involves actions on categories of sheaves, as Donu says in the comments.

Consider the following toy model: if $A$ is, say, a finite abelian group then its Pontryagin dual $\widehat{A}$ can be defined as the group $\text{Hom}(A, \mathbb{G}_m)$ of homomorphisms from $A$ into the multiplicative group $\mathbb{G}_m$ (say over the complex numbers, so $\mathbb{G}_m(\mathbb{C}) \cong \mathbb{C}^{\times}$, but any algebraically closed field of characteristic $0$ would do, or we could think in terms of Cartier duality). There is then a canonical pairing

$$A \times \widehat{A} \to \mathbb{G}_m$$

which is used, for example, to define the Fourier transform $L^2(A) \cong L^2(\widehat{A})$. In representation-theoretic terms these homomorphisms correspond to $1$-dimensional representations and give exactly the irreducible representations of $A$.

Abelian varieties $X$ also have duals $X^{\vee}$, but they're defined not in terms of maps into the multiplicative group $\mathbb{G}_m$ but in terms of line bundles, or equivalently in terms of maps into the classifying stack $B\mathbb{G}_m$ of line bundles (although we need to restrict to degree $0$ line bundles). There is again a canonical "pairing"

$$X \times X^{\vee} \to B \mathbb{G}_m,$$

namely the Poincaré bundle over $X \times X^{\vee}$, and it can be used to define the Fourier-Mukai transform $D(X) \cong D(X^{\vee})$ between derived categories of coherent sheaves. Among other things, what this analogy suggests is that the analogue of the "regular representation" for an abelian variety is its action on its derived category $D(X)$ by translation.

Answer 2

This is not an answer, more an extended comment. Though I am far from being an expert, I would have rather thought of an abelian variety (say $X$) to be analoguous to a vector (or projective) space (say $V$) and $\textrm{End}(X)$, the ring of isogenies of $X$, to be analogous to $\mathrm{GL}(V)$. Note that there is a notion of dual abelian variety which is quite close to the classical duality for projective space.

Furthermore assume that $X$ is simple (this is a very common hypothesis), that is $X$ contains no non-trivial abelian sub-variety. Then, any proper non-trivial homomorphism $f : X \longrightarrow X$ must be surjective with finite kernel. By a classical Theorem on abelian varieties, this implies that $f$ is an isogeny. So basically any interesting homomorphism from $X$ to $X$ is in $\mathrm{End}(X)$.