Suppose I have a smooth projective variety $X$ over $\mathbb{C}$ with $K_X$ semiample, and consider the fiber space $f:X\to Y$ given by $|\ell K_X|$, for some $\ell>0$ large, where $Y$ is a normal projective variety of general type.

Assume that the general fiber of $f$ is an Abelian variety, I would like to say that either there is a component of some singular fiber of $f$ which is uniruled, or otherwise all singular fibers (if any) are just smooth Abelian varieties with multiplicity $m>1$.

If $\mathrm{dim} X=2$ this statement is true thanks to Kodaira's classification of singular fibers of elliptic surfaces. In this case the singular fibers are either $mI_0, m>1$, or else have a component which is a rational curve. I am looking for a higher-dimensional analog of this statement, any reference (or counterexample) is greatly appreciated.