Let me slightly expand my comment from yesterday. Unfortunately, because of various time constraints, this will still be quite sketchy. Note that I'm only addressing the titular question. I have nothing to say about the automorphic aspects, since it is too far from what I know.
Let $X$ be a smooth projective variety over $\mathbb{Q}$. For simplicity assume that $X$ has rational point. I think there are various algebraic constructions of the Albanese in the 1950's literature, but I have to confess I've never gone through the details. So instead define $A=Alb(X) =Pic^0(Pic^0(X))$. After fiddling with the Poincare sheaf and the given rational point, you should get a Abel-Jacobi morphism $\alpha:X\to A$. I claim that this induces an isomorphism $$H^1(\overline{X}_{et},\mathbb{Q}_\ell)\cong H^1(\overline{A}_{et},\mathbb{Q}_\ell)$$ (where $\overline{X}= X\otimes \overline{\mathbb{Q}}$) necessarily compatible with the Galois action. To see this, base change up to $\mathbb{C}$, then by the comparison theorem, it suffices to get an isomorphism on the first singular (aka Betti) cohomology with $\mathbb{Q}$ coefficients. But the analytic construction of $Alb(X)$ gives this immediately because $$Alb(X)= H^0(X_{an},\Omega_X^1)^*/H_1(X_{an},\mathbb{Z})$$ So the rational first homologies coincide, now dualize.