At least over the complex numbers, $X$ is of general type by an old result of Ueno (I cannot locate it at the moment, I'll try later) that says the following:

Let $E$ be the biggest abelian subvariety of $A$ such that $X$ is invariant under translation by $E$, then $X/E$ is of general type.

At least over the complex numbers, $X$ is of general type by an old result of Ueno (see Damian's comment below) that says the following:

Let $E$ be the biggest abelian subvariety of $A$ such that $X$ is invariant under translation by $E$, then $X/E$ is of general type.

ADDED: (prompted by Damian's comment to the question). Pareschi and Popa [ Strong generic vanishing and a higher-dimensional Castelnuovo-de Franchis inequality. Duke Math. J. 150 (2009), no. 2, 269–285. ] have generalized the Castelnuovo-De Franchis inequality for surfaces and have proven that, if $X$ has no fibration onto a lower dimensional irregular variety, then $\chi(\omega_X)\ge q(X)-\dim X$, where $q(X)\ge \dim A$ is the irregularity. In particular, this applies if the Albanese variety of $X$ is simple.