In W. Neumann, J. Wahl, "Casson invariant of links of singularities", Comment. Math. Helv.,1990, Vol. 65, Issue 1, pp 58-78 some connection between the Casson invariant and the signature is formulated. Is there any heuristic explanation, why such a connection should be? The only reason I see is the Rokhlin invariant, which is defined by the signature and is reduction mod 2 of the Casson invariant. I would be happy to find some arguments in terms of Donaldson invariants or something like this.

Andras Nemethi and I have considerably generalized this result to rational homology $3$-spheres that are links of certain isolated surface singularities.

A rational homology $3$-sphere $M$ which is the link of an isolated surface singularity is equipped with a canonical spin-c structure. Associated to this spin-c structure is a Seiberg-Witten invariant which can be alternatively described in terms of the Casson-Walker invariant and the refined Reidemeister torsion as defined by Turaev; see this reference. If the $3$-manifold $M$ is an integral homology sphere, then it has a unique spin-c structure whose Seiberg-Witen invariant coincides with the Cason invariant.

In a sequence of papers (see here, here and here) A. Nemethi and I proved that for many classes of isolated singularities whose links are rational homology $3$-spheres, the Seiberg-Witten invariant of the above canonical spin-c structure of the link coincides with a certain algebraic geometric invariant of the singularity (essentially the geometric genus).

If the singularity happens to be a complete intersection, then this algebraic-geometric invariant can be expressed in terms of the signature of the Milnor fiber. Brieskorn integral homology spheres are of this type, and when specialized to this case, our result coincides Neumann and Wahl's older result.

I want to emphasize that our result applies even to situations when the singularity is not smoothable, i.e., Milnor fiber does not exist. (Most of the time the Milnor fiber does not exist.)

You can find a gentler introduction to this topic here.