Near the top of the second page of this paper, it is claimed that if two 4-manifolds $X,Y$ are homeomorphic, then their producs $X \times X, Y \times Y$ are diffeomorphic. Why is this true? It is trivially correct for 2-manifolds and lower dimensions. Is it also correct for 3-manifolds? What about higher dimensions?

References would be greatly appreciated as well.