Variation of centraliser in $\operatorname{GL}(n,\mathbb{Z})$ $\DeclareMathOperator\GL{GL}$Let $n$ be a positive integer $\geq 2$. The setting is that $K \in \GL(n,\mathbb{Z})$, and people are interested in understanding the centralizer:
$$
C(K)=\{ B \in \GL(n,\mathbb{Z}) \mathrel|B^{-1}KB=K  \}.
$$
For example, Eick, Hofmann, and O'Brien - The conjugacy problem in $\GL(n, Z)$ provides an algorithm computing the generator of a centralizer. In $n=2$ people have studied them extensively, there are a lot of references.
But I am interested in understanding the variant
$$C'(K)=\{ B \in \GL(n,\mathbb{Z}) \mathrel|B^{T}KB=K  \}.$$ I just change inverse to be the transpose. Obviously, $C'(K)$ is also a group. Moreover, the $\det(B)=\pm 1$ since one can take determinant on both sides. So I think this problem is much easier than the original problem. Although it seems like all the elements $B$ will determine a quadratic surface and we are asking for a set of solutions which is an arithmetic problem. The following two questions are interesting to me.

*

*One obvious question is that, given $K$, is $C'(K)$ a finite group?

*If so, can one find the maximal order of the group in terms of $n$? For example, in the original conjugate problem, people have shown that the maximal torsion order in $\GL(n,\mathbb{Z})$ is approximately $e^{n\log(n)^{1/2}}$ (Levitt - On the maximum order of torsion elements in $\GL(n, \mathbf Z)$ and $\operatorname{Aut}(F_n)$). A related MO question:
Maximal order of finite subgroups of $GL(n,Z)$.

Basically, I just want to ask, did people study this type of problem before instead of the conjugacy?
Any partial results or keywords are welcome.
 A: Here is an explicit example to show that the answer to the first question is "no" in general:
Let $m$ be an integer greater than $2$. Let $K = \left(\begin{array}{clcr} 1 & m\\0 & 1\end{array}\right)$ and let $B = \left(\begin{array}{clcr} 0 & -1\\1 & m\end{array}\right)$. Then $B^{T}KB = K$, but $B$ is not a matrix of finite order, ( its eigenvalues are real, but neither has absolute value $1$). The fact that $KBK^{-1} = (B^{T})^{-1}$ if your equality holds is what led to this example, for then, in particular,  $B$ and $B^{-1}$ must have the same eigenvalues.
Later edit: Note that when $K = I_{n}$, then $|C^{\prime}(K)| = 2^{n}n!$, which is probably the maximal possible order of such a group in the case that it is indeed finite (at least for large enough $n$). It is easy to prove (an argument of Blichfeldt), that every finite subgroup of ${\rm GL}(n,\mathbb{Z})$ has order  a divisor of $(2n)!$. The above group of order $2^{n}n!$ is sometimes known as the group of "signed permutation matrices".
A: A partial answer, which specialises YCor's comments in a more explicit way.
Let $K^+ = (K^T + K)/2$ and $K^- = (K^T - K)/2$. Then $C'(K)$ is the group of integral points of the intersection of the orthogonal group of the quadratic form $Q$ with Gram matrix $K^+$ and the symplectic form with Gram matrix $K^-$.
In particular:

*

*if $Q$ is positive or negative definite then $C'(K)$ is finite;

*if $K^-=0$ (i.e. $K$ is symmetric) then the converse of 1. holds;

*if $K^+=0$ (i.e. $K$ is antisymmetric) then $C'(K)$ is infinite (since every symplectic form is split).

You can check whether $Q$ is positive or negative definite by completing the squares of $Q$ and checking whether all the signs of the diagonal terms are the same.
As for the maximal order, assuming you want a bound only in terms of $n$: every finite subgroup of $GL_n(\mathbb{Z})$ stabilises a rational, positive definite quadratic form (by averaging). So being of the form $C'(K)$ is not a restriction at all for a finite subgroup of $GL_n(\mathbb{Z})$.
A: This is a very late comment, but for anyone still interested in this, I believe that YCor's idea can be made to work in practice using results in the following paper (see the introduction for proper attribution):
G. E. Wall, On the conjugacy classes in the unitary, symplectic and orthogonal groups, J. Austral. Math. Soc., vol. 3, 1963, 1-62.
I have not checked the details, but I believe that the algebraic group over $\mathbb{Q}$ can be described using Theorem 2.2.3 loc. cit. The first step is to compute $K^{-1} K^T$ and to decompose the space according to generalized eigenspaces for this endomorphism. The above theorem computes the graded pieces of a filtration of the group, the outermost piece being the largest reductive quotient. It is a product of orthogonal, symplectic and unitary groups (the latter in a generalized sense, that includes general linear groups), corresponding to eigenvalues $1$, $-1$ or something else for the above isomorphism. One can compute the signatures at the real place for the forms (symmetric, antisymmetric or hermitian) which occur and check the criterion given in YCor's comment for finiteness of arithmetic subgroups.
