Yes (Hirshon), even if $G_1, G_2$ are not finite.
Related: Can we ascertain that there exist an epimorphism $G\rightarrow H?$Can we ascertain that there exist an epimorphism $G\rightarrow H?$
Cancellation theorem for direct and other kinds of products between groupsCancellation theorem for direct and other kinds of products between groups