Let me slightly expand my comment from yesterday.  Unfortunately, because of
various time constraints, this will still be quite sketchy. Note that I'm only addressing the
titular question. I have nothing to say about the automorphic aspects, since it is too
far from what I know.

Let $X$ be a smooth projective variety over $\mathbb{Q}$. For simplicity assume
that $X$ has rational point. I think there are various algebraic constructions of
the Albanese in the 1950's literature, but I have to confess I've never gone through
the details. So instead *define* $A=Alb(X) =Pic^0(Pic^0(X))$. After fiddling with the
Poincare sheaf and the given rational point, you should get a Abel-Jacobi morphism
$\alpha:X\to A$. I claim that this induces an isomorphism 
$$H^1(\overline{X}_{et},\mathbb{Q}_\ell)\cong H^1(\overline{A}_{et},\mathbb{Q}_\ell)$$
(where $\overline{X}= X\otimes \overline{\mathbb{Q}}$)
necessarily compatible with the Galois action. To see this, base change up to $\mathbb{C}$,
then by the comparison theorem, it suffices to get an isomorphism on the first singular (aka Betti) cohomology with $\mathbb{Q}$ coefficients. But the analytic construction of $Alb(X)$ gives this immediately because
$$Alb(X)= H^0(X_{an},\Omega_X^1)^*/H_1(X_{an},\mathbb{Z})$$
So the rational first homologies coincide, now dualize.