This might be a wellknown problem but I am having trouble to find this. For square matrices $X, A, B,$ how to obtain the general solution for $X$, for the quadratic matrix equation $X A X^{T} = B$ ? What are the existence and uniqueness conditions for such solution?

2What sort of numbers make up the entries of the matrices? The problem is a bit harder if they are restricted to integers than if they are arbitrary complex numbers. – S. Carnahan♦ Oct 14 '11 at 11:13

I was looking for the case when matrices have real entries. Sorry, I should have mentioned that in the question. – Abhishek Halder Oct 18 '11 at 6:23
This relationship is called congruence, and for symmetric (or hermitian) matrices $A, B$ such an $X$ exists if and only if the matrices have the same inertia (the same number of positive, negative, zero eigenvalues). Fro general matrices congruence is much less well known, but it is quite well understood, see:
http://gauss.uc3m.es/web/personal_web/fteran/papers/20101.pdf

Thanks for pointing that out. I am wondering if $A$ and $B$ are symmetric positive semidefinite, then can we write $X$ explicitly in terms of $A$ and $B$? I am trying to get a result similar to Theorem 1 here: kb.osu.edu/dspace/handle/1811/22234?mode=full – Abhishek Halder Oct 18 '11 at 6:37

I very much doubt it. For $B=I,$ you are asking for a system of eigenvectors for $A,$ which is a question which has received a lot of attention over the last couple of centuries. If there were a closed form, that would be quite amazing. – Igor Rivin Oct 18 '11 at 9:23
I read the reference of Igor, but I have nothing shot about our equation !
There is a case that seems less difficult. If $A,B$ are symmetric complex, then the problem is essentially equivalent to solve $2$ equations of the form $ZZ^T=D$, that is, the case $A=I$ or $B=I$.
equation 1: $YY^T=A$ , equation 2: $ZZ^T=B$ , equation 3: $XY=Z$. If $A,B$ are invertible, then $Y,Z$ exist and $X=ZY^{1}$.In these conditions, can we find, at least in theory, all solutions ?
For Halder. The equation $XAX=B$ is easy to solve (at least when $A$ is invertible) because it can be rewritten $(XA)^2=BA$.
EDIT: Let $A,B\in\mathcal{M}_n(\mathbb{C})$. According to a result by Turnbull, Aitken (cited in the Igor's reference) our equation admits a solution iff $A,B$ are congruent iff there are invertible $P,Q$ s.t. $PAQ=B,PA^TQ=B^T$. Then, for generic $A,B$, the equation $XAX^T=B$ has no solutions. Thus, the "good" equation is $XAX^T=CAC^T$. Using my computer, I randomly choose real matrices $A,C$ ; I find that the algebraic set of complex solutions has dimension $1$ when $n=2$, $1$ when $n=3$, $2$ when $n=4$, $2$ when $n=5$.