# Matrices: characterizing pairs $(AB, BA)$

Let $A$ be an $m\times n$-matrix and $B$ an $n \times m$-matrix over the same field. Consider the matrices $C=AB$ and $D=BA$. It is probably well known (and not difficult to show) that the only difference between the canonic rational forms of $C$ and $D$ are nilpotent blocks (blocks with minimal polynomial $x^k$). (In particular, these compensate the different dimensions of $C$ and $D$.) I'm interested in the converse question:

Given an $m\times m$-matrix $C$ and an $n\times n$-matrix $D$, what are necessary and sufficient conditions that there exist matrices $A$ and $B$ such that $C=AB$ and $D=BA$?

One may assume without loss of generality that $C$ and $D$ are both nilpotent. I'm thinking of a characterization in terms of the Jordan normal forms of $C$ and $D$. These in turn are characterized by their block sizes. In fact, an equivalent version can be stated as a question on partitions:

Suppose $\lambda=(\lambda_1 \geq\lambda_2\geq \dotsc )$ and $\mu= (\mu_1 \geq \mu_2 \geq \dotsc)$ are partitions of the integers $m$ and $n$. When are there matrices $A$ and $B$ such that the blocks in the jordan normal forms of $AB$ and $BA$ belonging to the eigenvalue $0$ have sizes $\lambda_1, \dotsc$ and $\mu_1, \dotsc$, respectively?

These are not arbitrary, for example, the quotient of the minimal polynomials must be in $\{1, x^{\pm 1}\}$, meaning that $|\lambda_1-\mu_1| \leq 1$.
This problem seems so natural that I think it has been addressed somewhere (not in the linear algebra books I looked into, however), so in particular I would appreciate a reference.

EDIT: I have now seen that $|\lambda_i -\mu_i|\leq 1$ for all $i$ is sufficient (and this is easy, since we may assume $C$ and $D$ in Jordan form, and then reduce to the case of one Jordan block). I guess it's necessary, too. Does anyone know a reference for this? And are there any nontrivial mathematical applications of this situation?

"And are there any nontrivial mathematical applications of this situation?"

Yes, this is a very important construction in algebraic geometry and representation theory!

1. Algebraic geometry. The papers of Kraft and Procesi used this construction to analyze singularities of the closures of nilpotent orbits in the classical Lie algebras $\mathfrak{gl}_n, \mathfrak{sp}_{2n}, \mathfrak{o}_n.$ In particular, they proved that, in the case of $\mathfrak{gl}_n,$ these closures are normal varieties. Their proof is based on the relation $$\bar{\mathcal{O}}_{\lambda}=r\circ\ell^{-1}(\bar{\mathcal{O}}'_{\mu}). \qquad (*)$$ Here $\bar{\mathcal{O}}_{\lambda}$ is the closure of the conjugacy class of nipotent $n\times n$ matrices with partition $\lambda$ and $\bar{\mathcal{O}}'_{\mu}$ is the closure of the the conjugacy class of nipotent $m\times m$ matrices with partition $\mu,$ where $\mu_i=\operatorname{max}(\lambda_i-1,0)$; the diagram of $\mu$ is obtained from the diagram of $\lambda$ by removing the first column, so that $n-m=\lambda'_1.$ The maps $r$ and $\ell$ are $$r((A,B))=AB, \quad l((A,B))=BA$$ in the notation of the question. The papers of Daskiewicz, Kraskiewicz and Przebinda considered a more general situation: starting with $m$ and $n$ and a partition $\mu$ of $m,$ they proved that the formula (*) holds for a certain partition $\lambda$ of $n$. In other words, the algebraic variety $r\circ\ell^{-1}(\bar{\mathcal{O}}'_{\mu})$ is the closure of a single nilpotent orbit, without imposing additional assumptions on $m$ and $n;$ the proof involves careful combinatorial analysis, especially when $m>n.$

2. Representation theory. Without going into too much detail, this construction emerges in Roger Howe's theory of reductive dual pairs. The nilpotent orbits in question arise as the wave front sets or the associated varieties of representations of two classical groups occurring in the Howe duality correspondence. This is considered and exploited in various papers of J.-S. Li, T. Przebinda, P. Trapa, Nishiyama, Oshiai, and Taniguchi, and my own.

• Thank you for this very interesting answer! Using your notation, my question can be formulated as what $r\circ l^{-1} ( \mathcal{O}_{\mu} )$ is, where $\mathcal{O}_{\mu}$ is the conjugacy class of nilpotent $m\times m$-matrices with partition $\mu$. I'm (almost, I didn't check completely) sure now that it's those $\mathcal{O}_{\lambda}$ for which $|\lambda_i-\mu_i|\leq 1$ for all $i$. I guess this must be contained in the papers of Daszkiewicz et. al. you mention. I must confess, however, that I'm not familiar enough with the language used there to understand their results. Commented Mar 1, 2011 at 12:07
• By the way, reading your first point, I remembered that in Brieskorn's linear algebra(!) book (Lineare Algebra und analytische Geometrie, Band 2) there is some discussion of $\mathcal{O}_{\mu}$ and its closure (over the complex numbers with the usual topology of $\mathbb{C}^{m\times m}$). I don't have the book at hand now to see what exactly it's saying, however. Commented Mar 1, 2011 at 12:16
• Your criterion is basically correct, but needs to be stated with care: $\mathcal{O}_\lambda\subseteq r\circ\ell^{-1}(\mathcal{O}'_\mu)$' if and only if, after extending the partitions $\lambda$ and $\mu$ by a number of zero parts, $|\lambda_i-\mu_{\pi(i)}|\leq 1$ for all $i$ and some permutation $\pi$ of the indices. This follows from the description of the nilpotent orbits of the pairs $(A,B)$ and their behavior under the maps $\ell$ and $r$: these orbits are enumerated by the "$ab$-diagrams", which consist of strings of alternating $a$s and $b$s ($a, b, ab, ba, aba, bab, abab$, etc). Commented Mar 1, 2011 at 20:27
• The letter $a$ occurs $m$ times and the letter $n$ occurs $n$ times in the diagram; the map $\ell$ counts the $a$s in each string, discarding the $b$s, and the map $r$ counts the $b$s, discarding the $a$s. For example, $\ell(a, ab, bab, ababa)=(aaa, a, a, a)=(3,1,1,1)$ and $r(a, ab, bab, ababa)=(bb, bb, b)=(2,2,1,0);$ here $m=6, n=5.$ It seems plausible that the permutation $\pi$ is, in fact, not needed, but I'd need to think about it more carefully. By the way, the closures of $\mathcal{O}_{\lambda}$ in the usual topology and in the Zariski topology are the same. Commented Mar 1, 2011 at 20:42

By accident, I came across a note in the American Mathematical Monthly (C. R. Johnson and E. A. Schreiner, The Relationship between AB and BA, vol. 103 (MR)), where this is treated. It seems that this was first treated in the following article:

Harley Flanders, Elementary divisors of $AB$ and $BA$, Proc. Amer. Math. Soc. 2 (1951), p.871--874 (MR).

In fact, $\lvert\lambda_i-\mu_i\rvert\leq 1$ is the exact condition; this is sometimes called "Flanders's theorem". This gives a keyword to search for on Math Reviews, etc., but probably this has been rediscovered more than once. (Note that the titles above don't contain very specific keywords.)

• While looking up your references, I did an MSN search on the not-obvious-to-me query ti:"AB and BA"`, which seems to turn up a glut of articles on this and closely related questions (as well as an apparently unrelated concept in medicine of $ab$/$ba$ trials). Commented Sep 2, 2016 at 13:31
• All this is connected to the representations of the quiver with two vertices and one arrow from each vertex to the other. The indecomposable representations are known: look up the keyword "Contragredient equivalence". Commented Sep 2, 2016 at 18:26

This reminds me of the notion of strong shift equivalence in symbolic dynamics. Here one works with nonnegative integral matrices. Consider the equivalence relation on the set of nonnegative integral square matrices generated by saying that $AB$ is equivalent to $BA$ where $A$ is an $m\times n$ nonnegative integral matrix and $B$ is an $n\times m$ nonnegative interal matrix. This equivalence relation is called strong shift equivalence. The problem of deciding whether two matrices are strong shift equivalent is open to the best of my knowledge. This problem is equivalent to the conjugacy problem for shifts of finite type.

I realize that the problem has a lot of subtleties, but, first, we can naively count the number of constraints vs. the number of unknowns. For the former we have $n^2+m^2$ which come from the equations $C=AB, D=BA$, whereas, the number of unknowns is $2mn$ as both $A$ and $B$ have $nm$ elements. So we have $n^2+m^2 \geq 2mn$ and the equality saturates only for m=n. Thus unless any special assumptions are taken on the matrices C and D the decomposition does not take place.

• I know how to decide whether $C=AB$ has solutions when $C$ and $A$ are given, but that is not the question here. I'm interested in the case where quadratic matrices $C$ and $D$ are given, and I want to know how to decide whether the two equations $C=AB$ and $D=BA$ have a simultanous solution $(A, B)$. Most appealing would be a criterion in terms of the canonic rational forms of $C$ and $D$. Commented Feb 25, 2011 at 17:32
• sorry for misunderstanding, but let's keep counting the number of constraints vs. the number of unknowns. For the former we have $n^2+m^2$ which come from the equations $C=AB, D=BA$, whereas, the number of unknowns is $2mn$ as both $A$ and $B$ have $nm$ elements. So we have $n^2+m^2 \geq 2mn$ and the equality saturates only for $m=n$. Thus unless any special assumptions are taken on the matrices $C$ and $D$ the decomposition does not take place. Commented Feb 27, 2011 at 3:42