I think the answer is always yes. Considering the rational canonical form, one can deform a matrix continuously into a nilpotent matrix while maintining the sizes of the constituent companion matrices. Now keep the largest companion matrix fixed and scale down the remaining towards zero. We get a continuous path from our starting matrix to a conjugate $A$ to the direct sum $B_m$ of the zero matrix and the companion matrix of $x^m$. If the conjugacy is by a matrix of positive determinant, then it can be done via a path. But that is sufficient as there is a negative determinant matrix commuting with $B_m$.