The answer is affirmative (in Serre's formulation via principal homogeneous spaces) for proper, geometrically reduced, and geometrically connected schemes $X$ over any field $k$, giving a strong mapping property relative to working over all $k$-schemes. That is:
Theorem There exists a map $f:X \rightarrow E$ to a torsor for an abelian variety $A$ over $k$ such that for any $k$-scheme $S$ and any $S$-map $F:X_S \rightarrow T$ to a torsor for an abelian scheme $B$ over $S$, there exists a unique $S$-map $g:E_S \rightarrow T$ such that $g \circ f_S = F$.
Remark. The necessity of assuming geometric connectedness is illustrated in the Example at the end below. It is a useful weakening of geometric integrality (e.g., reducible semistable curves!), and Grothendieck's formulation in his Bourbaki expose appears to accidentally miss this necessary hypothesis.
Remark. In the above setting, there is a unique map of $S$-groups $A_S \rightarrow B$ making $g$ equivariant for their respective actions. Indeed, we claim the same for any $S$-map between torsors over $S$ for any two abelian schemes (no need for one of them to have arisen by base change from $k$), and by descent it suffices to prove such existence and uniqueness over an fppf cover of $S$. That reduces the task to the case of trivial torsors, so the claim is that if $B$ and $B'$ are abelian schemes over $S$ then any map of $S$-schemes $F:B \rightarrow B'$ (not necessarily respecting identity sections) is equivariant for a unique map between the abelian schemes. But $F = t_{b'} \circ f$ for $b' = F(e)$ and $f$ an $S$-homomorphism (with $t_{b'}:x \mapsto b'+x$), so all is clear.
The abelian variety $A$ turns out to be the dual of the maximal abelian subvariety of the (possibly non-reduced and non-proper) Picard scheme of $X$. The only literature reference I'm aware of is a Bourbaki expose by Grothendieck that treats "just" the case of geometrically normal proper $k$-schemes. In lieu of that, I give the proof of the more general result below, after reviewing what is in Grothendieck's paper (also identifying the step within which he missed the need for a geometric connectedness hypothesis).
To build up to this in stages, let's first go back to the ur-reference on this matter: the case of schemes that are proper and geometrically normal (and by necessity also geometrically connected, to be explained below) over any field is treated in Grothendieck's TDTE VI, Seminaire Bourbaki Expose 236, Cor. 3.2 and Theorem 3.3 (esp. part (iii), where $A$ refers to the unnamed abelian scheme in part (i), as in the discussion just before Theorem 3.3, and has nothing to do with $A$ in part (ii) that is not defined there but is the coordinate ring of a local scheme $S$).
In these results Grothendieck is assuming the existence of certain Picard schemes and dual abelian schemes (which of course he knew in many cases), and he is aiming to prove a stronger mapping property for the Albanese (over general $k$-schemes, not just over $k$ or field extensions thereof). The existence hypotheses that Grothendieck makes on Pic and on the dual abelian scheme in his statements of 3.2 and 3.3 were of course proved by him in the projective and geometrically integral case. Below we'll comment on the verification of such existence hypotheses more widely.
Note that in Corollary 3.2 when Grothendieck writes "normal et proper sur $k$" he means for "sur $k$" to qualify normality too (not just to qualify properness), with normality "over $k$" meaning by definition what is now called "geometrically normal" over $k$. One can see this intent from the use of 2.1(ii) in his proof (though it is also consistent with Cor. 6.7.8(vi) and Def. 6.8.1(vi) in EGA IV$_2$, where "normal over $k$" terminology is introduced with the same meaning).
The proofs in TDTE VI (as in other TDTE exposes) are sometimes just sketches, such as for 3.3(iii). The reason for his geometric normality hypothesis in 3.2 is to ensure via the valuative criterion for properness that ${\rm{Pic}}^0_{X/k}$ is proper, in which case he can use its finite etale $n$-torsion for all $n$ not divisible by ${\rm{char}}(k)$ to show that the underlying reduced scheme of this commutative finite type $k$-group scheme is a smooth $k$-subgroup scheme (hence an abelian variety).
The existence of Pic$_{X/k}$ as a locally finite type $k$-scheme for $X$ proper (not just projective) and geometrically reduced over a field $k$ was proved independently by Murre and Oort (if I remember correctly), though also recovered later by Artin as a special case of his existence results over general base schemes for Pic as a algebraic space quasi-separated and locally of finite presentation over the base (together with the fact that an algebraic space group locally of finite type over a field is a scheme, which is a consequence of Knutson's result that any locally noetherian quasi-separated algebraic space contains a dense open subspace that is a scheme, combined with careful translation arguments over finite extensions of the field).
In Theorem 3.3(iii) Grothendieck has to assume the existence of the dual abelian scheme. Over a field (all you need) this was of course known at that time in general since abelian schemes over fields are projective. In general it is due independently to Raynaud and Delinge, and is proved via algebraic-space techniques early in the book of Faltings and Chai. In general, the latter argument shows if $B \rightarrow S$ is an abelian scheme then the locally finitely presented algebraic space group ${\rm{Pic}}_{B/S}$ is $S$-separated (using valuative criterion) and the union of its fibral identity components is an open and closed $S$-subgroup ${\rm{Pic}}^0_{B/S}$ that is the dual abelian scheme, denoted $B^{\vee}$.
The upshot is that if a $k$-proper $X$ is geometrically reduced over $k$ then Grothendieck's existence hypotheses on Pic over $k$ and dual abelian schemes over any $k$-scheme are verified (and such objects over $k$ of course yield analogous representing objects after base change to any $k$-scheme). But what about the geometric normality hypothesis imposed in Cor. 3.2?
Grothendieck wants to build an abelian subvariety $C \subset P := {\rm{Pic}}_{X/k}$ such that any $S$-homomorphism to $P_S$ from an abelian scheme factors through $C_S$. The dual of such a $C$ will turn out to satisfy the desired Albanese property (for reasons we will explain below, using the further condition of geometric connectedness for $X$ over $k$). He follows the natural strategy of seeking conditions under which $P^0_{\rm{red}}$ is a smooth $k$-subgroup scheme of $P^0$ that is proper and hence is an abelian variety, in which case that would do the job (as one sees by considerations with relative schematic density of finite etale torsion levels in abelian schemes Zariski-locally on the base, or other reasons via $P^0/P^0_{\rm{red}}$). It is this strategy that leads him to impose the geometric normality condition on $X$.
But we can do better, by assuming only that $X$ is proper and geometrically reduced and geometrically connected. This involves applying the following result to ${\rm{Pic}}_{X/k}$.
Proposition. Let $G$ be a commutative group scheme locally of finite type over a field $k$. Let $C \subset G$ be the maximal abelian subvariety. For any $k$-scheme $S$ and abelian scheme $B$ over $S$, any $S$-homomorphism $B \rightarrow G_S$ factors through $C_S$.
Before we give the proof, we warn that $G_{\rm{red}}$ is generally not a $k$-subgroup of $G$ (when $k$ is not perfect and $G$ is not smooth), even if $G$ is geometrically connected; see Example 1.3.2(2) in Exp. VI$_{\rm{A}}$ of SGA3 for Raynaud's commutative affine example of such over any imperfect field.
Also, since any quotient of an abelian variety by a closed subgroup scheme is an abelian variety, we see that such a maximal $C$ exists for dimension reasons (and any abelian subvariety is contained in the identity component $G^0$ that is of finite type). Of course, one has no idea about the dimension of $C$ or even its non-triviality in general.
Proof. The main work is to show that the formation of such $C$ commutes with any extension of the ground field. Equivalently, after passing to $G/C$, we claim that if $C=0$ then $G_K$ contains no nonzero abelian subvariety for any extension field $K/k$. By Galois descent it is clear that the maximal abelian subvariety of $G_{k_s}$ vanishes. For any $K/k$, let $B \subset G_K$ be an abelian subvariety. We want to prove $B=0$. We can ramp up $K$ to be separably closed and hence contains $k_s$, so for the purpose of proving $B=0$ we can assume $k=k_s$.
For any $n > 0$ not divisible by ${\rm{char}}(k)$, $B[n]$ is a subgroup of $G_K[n]=G[n]_K$, and $G[n]$ is a finite etale $k$-group since it is killed by $n$. Thus, $G[n]$ is constant since $k=k_s$, so $B[n]$ descends uniquely to a $k$-subgroup $H_n \subset G[n]$. Let $H \subset G$ be the Zariski closure of the collection of $H_n$'s. The formation of $H$ commutes with extension of the ground field, so clearly $H_K = B$. Hence, $H$ is a $k$-subgroup of $G$ that is an abelian variety, so $H=0$ and hence $B=0$.
Now let $S$ be a $k$-scheme and $f:B \rightarrow G_S$ be an $S$-homomorphism from an abelian scheme $B$ over $S$. We want this to factor through $C_S$. By composing with the natural map to $(G/C)_S$ that represents the fppf quotient of $G_S$ modulo $A_S$, we can replace $G$ with $G/C$ so that $C=0$. Thus, for all $s \in S$ the $k(s)$-group scheme $G_s$ contains no nonzero abelian subvariety over $k(s)$ due to the compatibility with ground field extension discussed above.
In particular, the map $f_s: B_s \rightarrow G_s$ vanishes for all $s \in S$ since $B_s/(\ker f_s)$ is an abelian variety that is a $k(s)$-subgroup of $G_s$. But then by Corollary 6.2 in GIT (after passing to the case when $S$ is noetherian, as we may certainly do) it follows that $f=0$!
QED Proposition
Now we are ready to prove the Theorem stated at the start. The first thing we recall is that any torsor $E$ for an abelian variety $A$ over $k$ has ${\rm{Pic}}^0_{E/k}$ canonically isomorphic to the dual abelian variety $A^{\vee}$. Indeed, we can pick a finite Galois extension $k'/k$ so that $E(k')$ is non-empty, so $E_{k'} \simeq A_{k'}$ as torsors, well-defined up to $A_{k'}$-translation on $A_{k'}$. But the effect of $A_{k'}$-translation on $A_{k'}$ makes $A_{k'}$ act on the $k'$-group scheme ${\rm{Pic}}_{A_{k'}/k'}$, and that action is trivial since $A_{k'}$ is smooth and connected and ${\rm{Pic}}^0_{A_{k'}/k'}$ is an abelian variety (so its automorphism scheme as a $k'$-group is etale and thus receives no nontrivial homomorphism from an abelian variety). It follows that the pullback isomorphism $A^{\vee}_{k'} = {\rm{Pic}}^0_{A_{k'}/k'} \simeq {\rm{Pic}}^0_{E_{k'}/k'} = ({\rm{Pic}}^0_{E/k})_{k'}$ is independent of the choice of point in $E(k')$ that underlies the torsor isomorphism with $A_{k'}$. Thus, the composite pullback isomorphism is equivariant for the Galois descent data on both sides and hence it descends to a $k$-isomorphism $A^{\vee} \simeq {\rm{Pic}}^0_{E/k}$ that is independent of all choices.
The upshot of having the canonical isomorphism $A \simeq {\rm{Pic}}^0_{E/k}$ is that $A$ is not "extra data" in the statement of the Theorem, but rather is canonically determined by $E$. By using the etale topology in place of Galois theory, the exact same argument shows that for any torsor $T$ for an abelian scheme $B$ over a scheme $S$, the algebraic space ${\rm{Pic}}_{T/S}$ contains a unique open and closed $S$-subgroup ${\rm{Pic}}^0_{T/S}$ with (geometrically) connected fibers and that this $S$-subgroup is canonically isomorphic to the dual abelian scheme $B^{\vee}$, so in particular it is a scheme. (Any action of an abelian scheme on another abelian scheme over any base is trivial, due to triviality locally on the base for the action on the finite etale torsion levels that are collectively relatively schematically dense in the sense of EGA IV$_3$, 11.10.)
By the effectivity of Galois descent for (quasi-)projective $k$-schemes (such as $E$, whatever it may be) it suffices to prove the Theorem after making a preliminary finite Galois extension on $k$. Here we are using the earlier observation that any $g$ as in the Theorem determines a unique $S$-homomorphism $A_S \rightarrow B$ over which $g$ is equivariant. We are going to build the desired $A$ as dual to the maximal abelian subvariety $C$ of ${\rm{Pic}}_{X/k}$.
It is at this point that the need for geometric connectedness arises, and to clarify that we avoid using that condition for a moment. So
upon making a suitable finite Galois extension on $k$ we can arrange that the geometrically reduced (and hence generically smooth!) $k$-scheme $X$ has the form $\coprod X_i$ with each $X_i$ not only geometrically reduced but also geometrically connected and having a $k$-point $x_i \in X_i$.
Fix such choices of $x_i$ (an auxiliary device in the proof, not part of the statement of the Theorem).
Any $S$-map $F:\coprod (X_i)_S = X_S \rightarrow T$ to a torsor for an abelian scheme $B$ thereby has $T$ canonically identified with $B$ using the section $F((x_1)_S)$, and likewise for the sought-after universal $k$-map $f:\coprod X_i = X \rightarrow E$. Provided that there is only a single $X_i$ (i.e., the original $X$ is geometrically connected over $k$), our task is reduced to the analogue of the Theorem using abelian schemes and pointed maps (with $x_1 \in X(k)$) rather than torsors for abelian schemes. It is precisely at this step that geometric connectedness is essential, because otherwise making $(X_1)_S \rightarrow T$ a pointed map by identifying $T$ with $B$ via the image of $(x_1)_S$ would not have any impact on making the resulting $S$-maps $(X_i)_S \rightarrow B$ be pointed (and it is precisely here that counterexamples to the Theorem in the absence of geometric connectedness over $k$ creep in; see the Example at the end).
So now $X = X_1$ and we rename $x_1$ as $x$. There is a universal $x$-rigified line bundle $L$ on $X \times P$, so for any abelian scheme $B$ the pointed $S$-maps $F:X_S \rightarrow B = {\rm{Pic}}^0_{B^{\vee}/S}$ correspond exactly to isomorphism classes of $x$-rigidified line bundles $M$ on $X_S \times B^{\vee}$ that are trivialized along $X_S \times \{0\}$. The latter trivialization is unique upon requiring (as we may, via an $O(X_S)^{\times}$-scaling) it agrees with the effect of the $x$-rigidification along the $S$-point $(x_S,0)$. Thus, such data corresponds exactly to a pointed $S$-map $B^{\vee} \rightarrow {\rm{Pic}}_{X_S/S} = P_S$ ($P := {\rm{Pic}}_{X/k}$). Such a map is an $S$-homomorphism by Corollary 6.4 in GIT, so if $C$ denotes the maximal abelian subvariety of $P$ (via the Proposition above) then such maps correspond exactly to $S$-homomorphisms $B^{\vee} \rightarrow C_S$.
But by double-duality for abelian schemes, those in turn correspond exactly to $S$-homomorphisms $C^{\vee}_S \rightarrow B$. Unraveling, this procedure gives the result with $A := C^{\vee}$.
QED Theorem
Example. Consider $X = {\rm{Spec}}(k')$ for a finite etale $k$-algebra $k'$ having degree $> 1$, so this is geometrically reduced (and even geometrically normal) over $k$ but not geometrically connected. We claim that the conclusion of the Theorem cannot hold for such $X$ over $k$. Indeed, if there were a universal $f:X \rightarrow E$ over $k$
with the property as in the Theorem (for all $k$-schemes) then it would retain this property after scalar extension to a finite separable extension of $k$ that splits $k'$.
That is, it suffices to show the Theorem cannot hold when $X$ is a disjoint union of $d>1$ copies of ${\rm{Spec}}(k)$. Labeling the points as $x_1, \dots, x_d$, in the context of the mapping properties we can use the image of $x_1$ to trivialize the torsors and thereby drop it from consideration by using pointed maps to abelian schemes. That is, it suffices to show that if $Y$ is a disjoint union of $d-1>0$ copies of ${\rm{Spec}}(k)$ there is no $k$-map $f:Y \rightarrow A$ to an abelian variety over $k$ such that for any $k$-scheme $S$ any $S$-map $F:Y_S \rightarrow B$ to an abelian scheme $B$ over $S$ factors uniquely through an $S$-homomorphism $A_S \rightarrow B$. But $F$ is nothing other than the data of an $S$-point of $B^{d-1}$, and we aim to show that there is no abelian variety $A$ over $k$ equipped with a $k$-point $a \in A^{d-1}(k)$ with the magical property that every $S$-point of $B^{d-1}$ has the form $g^{d-1}(a_S)$ for a unique $S$-homomorphism $g:A_S \rightarrow B$. Even limiting ourselves to $S = {\rm{Spec}}(\overline{k})$ and varying $B$, it is clear no such amazing pair $(A, a)$ exists.
