Edit. There is a mistake in the answer below. It is possible that the fiber of the rational quotient has dimension $\geq 2$. I will try to revise the answer soon.
Original answer (including mistake).
If the anticanonical divisor class is numerically equivalent to $2F$, then by Mori's Bend-and-Break result, every point of your variety is contained in a rational curve. If the general fiber is a Calabi-Yau variety, then it is not uniruled. Thus, every rational curve containing a general point is not contained in the fiber. For transversal rational curves, deformation theory gives a lower bound on the dimension of the space of deformations containing the fixed (yet general) point: the lower bound is the anticanonical degree minus $2$. By Bend-and-Break again, that means that the minimal anticanonical degree of a transversal rational curve containing a general point is precisely $2$, i.e., the curve is a section of the fibration.
Moreover, the normal bundle is globally generated for every irreducible rational curve containing a general point, so the normal bundle is a trivial bundle. The following sentence is wrong. That means that the "rational quotient" of $X$ by these transversal rational curves is a $\mathbb{P}^1$-bundle over the quotient, and the fibers of the original fibration are mapping finitely to this quotient (away from codimension $2$). Why this is a mistake: in fact, there are rational quotients by such curves where the fiber dimension of the rational quotient map is strictly greater than $1$, e.g., cubic threefolds where the curve is a general line.
Since the fibers (of the projection to $\mathbb{P}^1$) are themselves Calabi-Yau, this should force the finite maps to be étale. Since Calabi-Yau varieties are simply connected, this should force the finite maps to be isomorphisms. So the rational quotient provides the projection to the fiber factor of the product of $\mathbb{P}^1$ and the fiber.