# Maximum of Tr$(ABA')$

Let $B$ be a fixed symmetric $M\times M$ matrix over the reals.

Let $A$ be an arbitrary $N\times M$ matrix over the reals.

I want to consider the problem of finding the extremal value of $\operatorname{Tr}(ABA^T) = \operatorname{Tr}(A^TAB)$ under the constraint that $AA^T$ is a fixed $N\times N$ matrix.

Can you give me a hint?

-
@Will, I believe the OP wants some restriction on $A$, as of now it says the product $AA^{T}$ has to be constant, which doesn't make sense because then the trace is uniquely determined. –  Gjergji Zaimi Aug 2 '11 at 1:55
If I understand the question, the answer should be $\sum \lambda_i \mu_i$ where $\lambda_1 \geq \lambda_2 \geq \dots$ (resp. $\mu_1 \geq \mu_2 \geq \dots$) is the non-decreasing sequence of eigenvalues of $BB'$ (resp. $AA'$). –  Mikael de la Salle Aug 2 '11 at 10:34
@ Mikael de la Salle, My expected answer (intuition) is not exactly but something like yours (the problem arises in a physical problem I'm working);...but why? –  dexter Aug 2 '11 at 10:48
@dexter: it took me several reads to finally (I think) understand what the question you are asking is. So I took the liberty to remove some unnecessary information (that my $B$ is of the form $XX^T$) for the problem, and put the constraint more prominent. –  Willie Wong Aug 2 '11 at 11:05
@dexter: the first thing to note is that $A A'$ and $A'A$ have the same nonzero eigenvalues. Then use the inequality $Tr(XY) \leq \sum \lambda_i \mu_i$ for positive definite matrices $X, Y$ with non-increasing eigenvalues $\lambda_i$ and $\mu_i$. People in matrix analysis certainly give a name to this inequality, but I do not know it. –  Mikael de la Salle Aug 2 '11 at 11:53