The answer is always yes. Indeed the set is path-connected.

Let $C(f)$ denote the companion matrix associated to the monic
polynomial $f$. Every matrix $A$ is similar to a matrix in rational
canonical form:
$$B=C(f_1)\oplus C(f_1 f_2)\oplus\cdots\oplus C(f_1 f_2,\cdots f_k)$$
where here $\oplus$ denotes diagonal sum. Then $m$ is the degree of
$f_1 f_2\cdots f_k$. Starting with $B$ deform each $f_i$ into a power
of $x$. We get a path from $B$ to
$$B'=C(x^{a_1})\oplus C(x^{a_2})\oplus\cdots\oplus C(x^{a_k})$$
inside $E_m$. There's a path from $B'$ in $E_m$ given by
$$(1-t)C(x^{a_1})\oplus (1-t)C(x^{a_2})\oplus\cdots\oplus C(x^{a_k})$$
ending at
$$B_m=O\oplus C(x^m).$$
Thus there is a path in $E_m$ from $A$ to $UB_mU^{-1}$
where $U$ is a nonsingular matrix. If $\det(U)\ne0$ then there is
a path in $GL_n(\mathbf{R})$ from $U$ to $I$ and so a path in $E_m$
from $A$ to $B_m$. If $m< n$ then there is a matrix $V$ of negative determinant
with $VB_m V^{-1}=B_m$ so that we may take $U$ to have positive determinant.

The only case that remains is when $m=n$. In this case $E_m$ contains
diagonal matrices with distinct entries, and each of these commutes
with a matrix of negative determinant.