Going out on a limb (I may well be messing up badly), I think the answer is _yes_ at least for the CW structure version of the question.

**Proof:**

Choose a finite presentation of $F$ with generating set $G$ and relation set $R$, and consider the induced $\pi_1$-isomorphism $X = (S^1)^{\vee G} \cup_{(S^1)^{\vee R}} \ast \to BF$. Note that $X$ has finitely many cells. I _believe_ that the rank over $\mathbb Z[F]$ of $H_\ast(\tilde X ; F)$ grows polynomially in $\ast$ (if it doesn't, then we should have a homology obstruction, giving a negative answer to the question), and that one can simply attach cells one-by-one, starting with $X$, building up a space $X'$ whose universal cover has the same $\mathbb Z[F]$-homology as $BF$ and conclude by homology Whitehead (with coefficients) that $X' \to BF$ is a weak homotopy equivalence.