Determining if a matrix is orthogonal

Let g be an element of $GL_n(\mathbb C)$. We know that there are orthogonal groups $O(\beta)=\{X\in GL_n(\mathbb C) \mid X^t\beta X=\beta\}$ for any $\beta$, invertible symmetric matrix. Though these groups are conjugate since all symmetric bilinear forms over $\mathbb C$ are equivalent. In literature one can find a way to determine when a matrix $X$ satisfies $X^tX=I$ thus an element of the orthogonal group $O(I)$.

My question is as follows: Is there a way to determine if an invertible matrix X belongs to $O(\beta)$ for some $\beta$. To me it seems that I have to check for all conjugates. Sure enough there will be an easier way out!

More precisely I want to know the following: Given $X$ an invertible matrix how to determine $\beta$, an invertible symmetric matrix, so that $X^t\beta X=\beta$?

• Maybe my intuition is wrong but if we think of $f(\beta) = X^t \beta X$ as a function then basically we want to know if a fixed point exists for which $f(\beta) = \beta$. My first thought goes to some iterative method, start by assuming that $\beta_1 = I$ and computer $\beta_2 = f(\beta_1)$ then plug that value and try again if a fixed point exists then this iteration would/could converge. – Pushpendre Jul 2 '15 at 5:18
• For the first part: Since you need $\beta X\beta^{-1} = (X^{t})^{-1}$, it is certainly necessary that $X$ and $X^{-1}$ have the same characteristic polynomial. – Geoff Robinson Jul 2 '15 at 7:51
• right! we need to look at self-reciprocal polynomials. And some clever condition does the job. – Anupam Singh Jul 2 '15 at 8:03

There is a complete characterization of matrices that belong to at least one orthogonal group. It reads as follows over any arbitrary field $\mathbb{F}$ with characteristic different from $2$ (with algebraic closure denoted by $\overline{\mathbb{F}}$:

Given a matrix $M \in \mathrm{GL}_n(\mathbb{F})$, there exists an invertible symmetrix matrix $\beta$ such that $M^T \beta M=\beta$ if and only if, for every $\lambda \in \overline{\mathbb{F}} \setminus \{0,1,-1\}$ and every positive integer $k$, one has $\mathrm{rk}(M-\lambda I_n)^k=\mathrm{rk} (M-\lambda^{-1} I_n)^k$ and, for each one of the (possibly absent) eigenvalues $1$ and $-1$ and every positive integer $k$, there is an even number of Jordan cells of size $2k$ in the Jordan reduction of $M$.

Moreoever, if you have access to the Jordan reduction of $M$ and the above conditions are satisfied, then coming up with an explicit solution $\beta$ is not difficult.

This characterization has been known for a very long time. See my preprint http://arxiv.org/abs/1008.4458 for a recent account using elementary methods.

Consider the entries of $X^t \beta X - \beta$ as linear equations in the entries of the symmetric matrix $\beta$, and solve. For $n \times n$ matrices these are $(n+1)n/2$ equations in $(n+1)n/2$ unknowns. If there is a nontrivial solution, compute the rank of $\beta$ (using generic values for the free variables).

EDIT: It probably should be mentioned that this is a similarity invariant, i.e. if $\beta$ works for $X$ then $S^t \beta S$ works for $S^{-1} X S$ where $S$ is any invertible matrix.

In the cases $n=4$ and $n=5$, I tried companion matrices of self-reciprocal polynomials $p(x) = 1 + a_1 x + a_2 x^2 + a_1 x^3 + x^4$ and $p(x) = 1 + a_1 x + a_2 x^2 + a_2 x^3 + a_1 x^4 + x^5$ respectively. It turns out that for $n=4$ there are invertible solutions $\beta$ iff $a_1 \ne \pm (1 + a_2/2)$, and for $n=5$ iff $a_1 \ne -1 - a_2$.

Interesting question !

Let us denote $L_X:\beta\mapsto X^T\beta X$. This is an endomorphism of ${\bf Sym}_n({\mathbb C})$.

Lemma. The characteristic polynomial of $L_X$ is $$\prod_{i\le j}(t-\mu_i\mu_j)$$ where $\mu_1\ldots,\mu_n$ are the eigenvalues of $X$.

Sketch of the proof : the characteristic polynomial has degree $n(n+1)/2$. Suppose that $X$ is diagonalisable, and that the products $\mu_i\mu_j$ are pairwise distinct. Then each $\mu_i\mu_j$ is an eigenvalue, associated with an eigenvector $\ell^T_i\ell_j+\ell^T_j\ell_i$ where $\ell_iX=\mu_i\ell_i$. So these are all the eigenvalues and the formula is correct in this case. Because the characteristic polynomial of $L_X$ is also a polynomial in the entries of $X$, as well as the product indicated above, the equality stands in the closure (topological or Zariski) of the special case, which is ${\bf M}_n(\mathbb C)$.

Now, what are we looking for ? An eigenvector $\beta$, non-singular and associated with the eigenvalue $1$. Thus we need $\mu_i\mu_j=1$ for some pair. But not only for some, because $\ell^T_i\ell_j+\ell^T_j\ell_i$ is singular (unless $n=2$). What we really need is enough multiplicity that we can pick a non-singular eigen-matrix. If $X$ is diagonalizable, this means that the eigenvalues be associated pairwise: $$\mu_1\mu_n=\mu_2\mu_{n-2}=\cdots=1.$$ Notice that if $n$ is even, this yields the necessary condition that $\det X=1$. It was obvious a priori that $(\det X)^2=1$, but actually the case $-1$ is excluded. This ressembles the well-known fact that symplectic matrices have determinant $+1$.

Edit. A more direct and definitive way to arrive to this conclusion : if $\beta$ exists, then $X^{-T}=\beta X\beta^{-1}$, that is $X^T$ is conjugated to $X^{-1}$. This proves that $\lambda\mapsto\lambda^{-1}$ is an involution of the spectrum of $X$. Conversely, suppose that $X^t$ and $X^{-1}$ are conjugated ; can we choose a symmetric conjugation matrix ? The answer is no, as shown by the counterexample $$X=\begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}.$$

• The eigenvalues do not all need to be paired up; it is possible to have $\det X=-1$. For instance, it is trivial to see that the scalar matrix $-I$ is always orthogonal. More generally, an eigenvalue which is $\pm 1$ can be "paired with itself" in when constructing a nonsingular $\beta$. – Eric Wofsey Jul 3 '15 at 17:57
• I wonder if we can make an if and only if statement using eigen values of X for a particular $\beta$. For example it is great to know that $\beta$ has form $l_i^tl_j + l_j^t l_i$. – Anupam Singh Jul 6 '15 at 11:23

Not quite an answer: If you replace "symmetric" by hermitian, then, and transpose by conjugate transpose, then, since every hermitian matrix can be written by as $\beta = A^* A,$ then $X^* \beta X = \beta$ implies that a conjugate of $X$ lies in the unitary group, which is necessary and sufficient. That is the same as saying that the eigenvalues of $X$ are of modulus one (and the matrix is diagonalizable).The same works with "symmetric" and "transpose" but over $\mathbb{R}.$

ADDITION for symmetric matrices, there is a canonical form (due to Horn and Sergeichuk: Canonical forms for complex matrix congruence and ∗congruence Roger A. Horn a,∗ , Vladimir V. Sergeichuk b [Linear Algebra and Applications, 2006]. ) See, equations (7) and on in the paper. There are three types of blocks, and clearly a matrix is in the orthogonal group of some complex symmetric matrix if it is conjugate to a direct sum of matrices in the orthogonal groups of the three block types.

• No, first you mean "modulus 1" instead of "roots of unity", second you also need the matrix to be diagonalizable over the complex numbers. – YCor Jul 2 '15 at 9:14
• @YCor Yes, correct. That's what happens when you do this too late at night :) – Igor Rivin Jul 2 '15 at 12:56
• The symmetric case seems to be much more complicated, though. For instance, it is easy to see that a diagonal matrix is in some $O(\beta)$ iff its eigenvalues other than $\pm1$ come in inverse pairs (see, for instance, Geoff Robinson's comment). But for $n\geq 3$, a complex orthogonal matrix need not be diagonalizable, and it seems nontrivial to determine what Jordan normal forms can occur depending on $n$. – Eric Wofsey Jul 2 '15 at 15:18
• The paper "The Jordan canonical forms of complex and skew-symmetric matrices" by Horn and Merino classifies which Jordon blocks are similar to an orthogonal matrix with $\beta=I$. However I would like to identify $\beta$ as well in the process. – Anupam Singh Jul 3 '15 at 5:24