Freedman’s theorem shows that all 3-dimensional homology spheres bound topologically a contractible 4-manifold. It is well known that the Poincaré homology sphere does not bound a smooth contractible 4-manifold. Is there other explicit examples or is there an invariant of homology sphere characterizing this property(whether or not smoothly bound a contractible 4-manifold)?
2 Answers
There are certainly lots of obstructions coming from gauge theory, for instance Frøyshov's $h$-invariant or Ozsváth and Szabó's $d$-invariant. If an integer homology sphere $P$ has $d(P) \neq 0$, then $P$ does not even bound a rational homology ball. There are $\mathbb{Z}/2\mathbb{Z}$ refinements (more precisely, obstructions to bounding spin rational homology balls).
In addition to this, and closer to your question: using periodic ends and Seiberg–Witten theory, Taubes proved that if $Y$ is a homology 3-sphere that bounds a negative definite 4-manifold with non-diagonal intersection form (e.g. $Y$ is the Poincaré sphere, which bounds the $E_8$-plumbing of spheres), then $Y \# {-Y}$ does not bound a contractible 4-manifold. (It does bound an integer homology 4-ball, namely $(Y\setminus B^3)\times [0,1]$.) As far as I know, there is no single invariant capturing this information. Certainly there is no single invariant capturing whether a 3-manifold bounds an integer homology 4-ball.
Some keywords you might care about are: (integer or rational) homology cobordism group, homology cobordism invariants, Heegaard Floer correction terms (or variants thereof, like involutive Floer homology).
Freedman proved that any integral homology 3-sphere topologically bounds a contractible, compact 4-manifold. If we start with an integral homology 3-sphere $\Sigma$ and double the associated contractible 4-manifold along $\Sigma$, then the resulting space is a homotopy 4-sphere, which is homeomorphic to $S^4$ (due to the topological 4D Poincare conjecture). So I think the core of the posted question is whether $\Sigma$ smoothly embeds in $S^4$. (It's easy to see that every homology sphere admits a locally flat topological embedding in $S^4$) There are Brieskorn homology spheres which smoothly embeds but some Brieskorn homology spheres do not embed. If I remember correctly, it's still unclear which Brieskorn homology sphere embeds and which does not. On P31 in Budney-Burton, they listed several Brieskorn homology spheres which do not embed in $S^4$ since the Rochlin invariant or d-invariant (which is mentioned in Marco Golla's answer) is not zero. However, Budney-Burton also provided Brieskorn homology spheres do embed in $S^4$ smoothly. See Theorem 2.13 or P24. I don't know if there are other classified Brieskon homology spheres which are not listed in their survey.
If one is willing to expand the roster of candidates, Marco Golla has already given a nice answer in general. A weaker question than the posted one could be when an integral homology sphere can arise as the boundary of an acyclic 4–manifold. There are some recent research surrounding this, e.g. Homology spheres bounding acyclic smooth manifolds and symplectic fillings