The arguments of Serre can be in fact made to work over any separably closed field. The result in the general case can then be deduced using Galois descent. Details can be found in Section 2 and the appendix of:

Olivier Wittenberg - On Albanese torsors and the elementary obstruction.

This is in particular shows the existence of the Albanese torsor and Albanese variety for any geometrically integral variety $X$ over any field $k$.

Note that when dealing with singular varieties, the Albanese variety is usually defined to be a semi-abelian variety, rather than just an abelian variety.

The arguments of Serre can be in fact made to work over any separably closed field. The result in the general case can then be deduced using Galois descent. Details can be found in Section 2 and the appendix of:

Olivier Wittenberg - On Albanese torsors and the elementary obstruction.

This is in particular shows the existence of the Albanese torsor and Albanese variety for any geometrically integral variety $X$ over any field $k$.

Note that when dealing with non-proper varieties, the Albanese variety is usually defined to be a semi-abelian variety, rather than just an abelian variety.