The matrix
$0\ 1\ 0$
$1\ 0\ 0$
$0\ 0 \-1$
in $SL_3(\mathbb Z)$ is not conjugate to any block sum of an $SL_2(\mathbb Z)$ matrix and $+1$. And of course it is not conjugate to any block sum of an $SL_2(\mathbb Z)$ matrix and $-1$, either.
For $GL_3$ and $GL_2$ the answer is yes.
EDIT This last statement is wrong; see Geoff Robinson's comments.