First, you probably also want to fix the action of not just $\pi_1(X_i)$ on H_2$but$H_2(X_i)$. H_2(M, {\mathbb Z}[\pi_1(M)])$ together with the intersection form on this group. With this in mind, if $M$ is a closed 4-manifold whose fundamental group is infinite cyclic, then Freedman-style classification is indeed available for $M$, but requires extra work which was done by Stong and Wang in "Self-homeomorphisms of 4-manifolds with fundamental group ${\mathbb Z}$", where they corrected some errors in the book of Freedman and Quinn. (Wang may have done this earlier in his unpublished thesis.) In particular, in this setting, you get only finitely many topological types of the manifolds $M$. Stong and Wang also prove that a self-homeomorphism of such $M$'s are "almost" determined (up to pseudoisotopy) by its action on $H_2$, regarded as a $\pi_1(M)$-module. This H_2(M, {\mathbb Z}[\pi_1(M)])$.This result might take care of manifolds whose fundamental group is${\mathbb Z}\times {\mathbb Z}_2$(by considering free actions of topological involutions on manifolds$M$above). However, you would have to check if two pseudo-isotopic involutions are topologically conjugate. 1 First, you probably also want to fix the action of$\pi_1(X_i)$on$H_2(X_i)$. With this in mind, if$M$is a closed 4-manifold whose fundamental group is infinite cyclic, then Freedman-style classification is indeed available for$M$, but requires extra work which was done by Stong and Wang in "Self-homeomorphisms of 4-manifolds with fundamental group${\mathbb Z}$", where they corrected some errors in the book of Freedman and Quinn. (Wang may have done this earlier in his unpublished thesis.) In particular, in this setting, you get only finitely many topological types of the manifolds$M$. Stong and Wang also prove that a self-homeomorphism of such$M$'s are "almost" determined (up to pseudoisotopy) by its action on$H_2$, regarded as a$\pi_1(M)$-module. This result might take care of manifolds whose fundamental group is${\mathbb Z}\times {\mathbb Z}_2$(by considering free actions of topological involutions on manifolds$M\$ above). However, you would have to check if two pseudo-isotopic involutions are topologically conjugate.