I found some very old notes in which I worked out some restrictions on the base space for elliptically fibered Calabi-Yau 3-folds, part of it will apply in any dimension. There are some references to results that I cannot find now, I would be very interested myself if anyone can provide the details.
We have $\pi_\ast\mathcal O_X = \mathcal O_B$. The Leray spectral sequence of the fibration is
$$E_2^{pq} = H^p(B,R^q\pi_\ast\mathcal O_X)\Rightarrow H^{p+q}(X,\mathcal O_X).$$
Since the dimension of the fibers is 1, we have that $R^q\pi_\ast\mathcal O_X = 0$ for $q > 1$. Since these sheaves are coherent, we also have that $H^p(B,R^q\pi_\ast\mathcal O_X) = 0$ for $p > \text{dim}(B)$.
According to my notes it is a result of Kollár (that I didn't manage to find back) that $d_2$ is 0, so that $E_2 = E_\infty$. This may only hold for dimension 3.
It immediately follows that
- $H^0(B,\mathcal O_B) = k$
- $H^1(B,\mathcal O_B) = H^0(B, R^1\pi_\ast\mathcal O_X) = 0$
- $H^2(B,\mathcal O_B) = H^1(B, R^1\pi_\ast\mathcal O_X) = 0$
- $H^2(B, R^1\pi_\ast\mathcal O_X) = k$
We also have an inequality of Kodaira dimensions (due to Ueno? I'm not sure, but I guess the result is well-known, references welcome)
$$0 = k(X) \ge k(X_b) + k(B) = k(B)$$
where $X_b$ is the generic fiber, so that $k(B) = 0$ or $k(B) = -\infty$.
In the three dimensional case it follows (by the classification of surfaces) that $B$ is rational, ruled over $\mathbb P^1$ or Enriques.
EDIT I found in this article by Chen and Zhang that the result that I attributed to Ueno above was actually proved by Viehweg for varieties over $\mathbb C$, which they generalize to positive characteristic under some assumptions.
EDIT (Apologies for yet another edit): From the classification of surfaces we actually see that ruled surfaces are ruled out as well, so the base has to be rational or an Enriques surface (namely both the geometric genus and the irregularity, which are $h^{2,0}$ and $h^{1,0}$, are 0). Finally, according to the comment by Jason Starr, at least over $\mathbb C$ $B$ must be simply connected, while an Enriques surface has $|\pi_1(B)| = 2$, so that $B$ must be rational.