Given a map $f : S \to M^4$ from a compact closed not necessarily connected oriented surface to a compact oriented 4-manifold, such that $f_*([S])$ is zero in $H_2(M)$, is there a compact oriented 3-manifold $W$ with boundary $S$ and a map $F : W \to M$ that extends $f$?

Yes, there is. This follows from the definition of the (singular oriented) bordism group $\Omega_2(M)$ and the fact that the natural map $\Omega_2(M)\to H_2(M;\mathbb{Z})$ is an isomorphism. In fact the conclusion holds for much more general spaces $M$ (such as CW-complexes).

In more detail, $\Omega_2(M)$ denotes equivalence classes of continuous maps $f: S\to M$, where $S$ is a closed oriented surface (not necessarily connected), under the relation of *bordism*: two such maps $f_1: S_1\to M$ and $f_2: S_2\to M$ are declared bordant if there is a $3$-manifold $W$ with boundary $\partial W = S_1\sqcup S_2$ and a map $F:W\to M$ extending $f_1\sqcup f_2: S_1\sqcup S_2\to M$ (everything up to diffeomorphism). This defines an abelian group with addition given by disjoint union. The zero element is represented by any $f:S\to M$ which bounds a map from a $3$-manifold (as in your question).

The natural map $\Omega_2(M)\to H_2(M;\mathbb{Z})$ sends $[f:S\to M]$ to $f_\ast([S])$. The easiest (but perhaps not the most elementary) way to see that this is an isomorphism is to examine the Atiyah-Hirzebruch spectral sequence for bordism (see Conner and Floyd's "Differentiable Periodic Maps", section 7), armed with the additional data that $\Omega_1 = \Omega_2 = 0$, i.e all oriented $1$ and $2$-manifolds bound.