Thanks for your wonderful answer! It clears part of my confusion up. I suppose another complication is that certain classes on the boundary of the circular nef cone might actually be effective right? For example, the classes $v, h, \Delta, \Gamma$ are all on the boundary. And presumably any effective curve of square zero lies on the boundary.

For a curve $C$ Mumford introduces the subgroup $H(C)$ of $A$ consisting of all $a \in A$ such that translation by $a$ preserves $C$ (as a curve, not just its class). So is the idea that $\text{Chow}_{\beta}(A) = (|L| \times A)/\sim$ where $(C, a) \sim (C,b)$ if $a-b \in H(C)$?

@DavidLoeffler Is there something in the literature which covers what I'm asking about? Ideally, I'd like to be able to look these weight 3 forms up, if it's that easy.

Interesting, I didn't know about this nice connection to extremal K3s. Thanks! So you're expecting that the modular form for $A$ and it's Kummer K3 should be the same? Also, do you mind if I clarify what you mean by $S^{2}$? I was guessing symmetric product, but I thought we wanted specifically 2-dimensional Galois representations.

@reuns Someone can probably give a better comment than me, but if a $d$-dimensional variety is modular, I think it is expected that the weight should be $d+1$. For example, elliptic curves correspond to weight 2 forms, and rigid Calabi-Yau threefolds give weight 4 forms. So K3 and abelian surfaces should be weight 3.

@Kimball Right, it's in $\text{Sp}_{4}(\mathbb{R})$. This $K^{*}(N)$ formed using $V_{N}$ seems standard, for example see page 5 of (math.unt.edu/~schmidt/dimension_formulas/papers/…)

@WillSawin Great that makes sense. Thanks a lot. If it matters, I would accept it if you wrote this into an answer.

@WillSawin I suppose, but is it clear the refinement will be Whitney? Or is it not true that a constructible sheaf must be locally constant over a Whitney stratification? I've been confused whether you need Whitney in the definition of constructible, because I've seen it both ways.