Kernel of $AB$ if $[A,B]=0$ and $AB\neq0$? [closed]

Question

I have found similar results here and mathematics stack exchange but they all imposed specific conditions that don't suit this problem in particular. The problem is as follows.

Let A,B be square $n\times n$ matrices such that

1) $[A,B]=0$

2) $AB\neq0$

3) $A^2=B^2=0$

4) $\mbox{Ker}(A)\cap\mbox{Ker}(B)\neq\{0\}$

Then $\mbox{Ker}(AB)=\mbox{Ker}(A) + \mbox{Ker}(B)$.

It seems to be true for all of the examples I can come up with, and one set inclusion is obvious but I don't know how to prove the non-trivial direction to obtain equality. Any advice or references to where this has been solved?

Thanks!

Answer 1

The result does not hold. Here is a counterexample: take $X=\begin{bmatrix} 1 \\ 0 \end{bmatrix}$, $E=\begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}$, $F=\begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix}$, and consider the matrices $$A:=\begin{bmatrix} [0]_{2 \times 2} & [0]_{2 \times 2} & I_2 & [0]_{2 \times 1} \\ [0]_{2 \times 2} & [0]_{2 \times 2} & [0]_{2 \times 2} & X \\ [0]_{2 \times 2} & [0]_{2 \times 2} & [0]_{2 \times 2} & [0]_{2 \times 1} \\ [0]_{1 \times 2} & [0]_{1 \times 2} & [0]_{1 \times 2} & 0 \end{bmatrix}$$ and $$B:=\begin{bmatrix} [0]_{2 \times 2} & E & [0]_{2 \times 2} & [0]_{2 \times 1} \\ [0]_{2 \times 2} & [0]_{2 \times 2} & F & [0]_{2 \times 1} \\ [0]_{2 \times 2} & [0]_{2 \times 2} & [0]_{2 \times 2} & X \\ [0]_{1 \times 2} & [0]_{1 \times 2} & [0]_{1 \times 2} & 0 \end{bmatrix}$$ of $M_7(\mathbb{F})$. One easily checks that $A$ and $B$ all have the required properties. However, $Ker(A)+Ker(B)=\mathbb{F}^5 \times \{0\}$ whereas $Ker(AB)=\mathbb{F}^6 \times \{0\}$.

Answer 2

Here is an intuitive way to see why this must be false. You want to establish an equality, so your (dependent) inequalities 2) and 4) are suspect; inequalities are seldom useful in proofs of equalities.

You have no doubt found examples where all conditions hold, for instance the $4\times4$ matrices $A_0=E_{1,2}+E_{3,4}$ and $B_0=E_{1,3}+E_{2,4}$ (with $E_{i,j}$ elementary matrices) are nilpotent of order$~2$, they commute, have non-trivially intersecting kernels and $A_0B_0\neq0$. They also satisfy the conclusion.

On the other hand taking $A_1=B_1$ any matrix of nilpotency order $2$ the conclusion will be false, but so will conditions 2) and 4). I would bet you put in those conditions mainly to beat these obvious counterexamples. But adding conditions that are unlikely to help with a proof is not such a good idea.

Since those conditions are just inequalities, it suffices to take block diagonal matrices $A,B$ with diagonal blocks $A_0,A_1$ respectively $B_0,B_1$ to get a genuine counterexample. You should be able to find a $6\times6$ counterexample in this manner.