On the isometry group of a self cartesian product of a Riemannian space Let $X$ be a complete Riemannian space. Let us denote by $Iso(X)$ the group of isometries of $X$. It is a well-known fact that the group $Iso(X)$, when endowed with the compact-open topology, is a Lie group. Let $X\times X$ be the cartesian prodcut of $X$ with itself endowed with the product metric. Let 
$G:=(Iso(X)\times Iso(X))\rtimes S_2$, where $S_2$ is the symmetric group of order $2$ which permutes the two coordinates. We have a natural inclusion of $G\leq Iso(X\times X)$.
Q: Is there a general criterion on $X$ which allows us to determine when is $G=Iso(X\times X)$ ?  
 A: There is a simple answer in the case that $X$ is simply connected.
In this case, your claim is true if and only if $X$ is not flat and does not split as a Riemannian product.
"$\Longleftarrow$". If $X$ is as above, each isometry of $X\times X$ has to permute the de Rham factors. There are only two, so they are swapped or they aren't.
"$\Longrightarrow$". If there is a flat de Rham factor $\mathbb R^k$, then the isometry group of $\mathbb R^{2k}$ is larger than the group in the question. If there is more than one de Rham factor, an isometry can choose to swap the corresponding de Rham factors in the two copies of $X$ for each de Rham factor separately, so again the group is larger.
EDIT:
In the nonsimply connected case, a similar statement holds: the isometry group of $X\times X$ is exactly $\mathrm{Iso}(X)^2\rtimes(\mathbb Z/2)$ if and only if $X$ is indecomposable in the sense of Eschenburg and Heintze and not isometric to flat $\mathbb R^k$ for $k\ge 1$. The argument is as above, using the main theorem of op. cit. (thanks to Holonomia for the hint).
