The matrix

$0\ 1\ 0$

$1\ 0\ 0$

$0\ 0 \-1$

in $SL_3(\mathbb Z)$ is not conjugate to any block sum of an $SL_2(\mathbb Z)$ matrix and $+1$. And of course it is not conjugate to any block sum of an $SL_2(\mathbb Z)$ matrix and $-1$, either.

For $GL_3$ and $GL_2$ the answer is yes.

EDIT  This last statement is wrong; see Geoff Robinson's comments.