$\newcommand\norm[1]{\lVert#1\rVert} \newcommand\opnorm[1]{\norm{#1}_{\text{op}}} \newcommand\Frnorm[1]{\norm{#1}_{\text F}}$Define \begin{align*} S_t=\{ (A,B)\in\mathbb{R}^{n\times n}\times\mathbb{R}^{n\times n} :{} & \frac{\Frnorm{A - B}}{\sqrt{n}}\leq\sqrt{t}, \frac{\Frnorm A}{\sqrt{n}}\geq1-t, \frac{\Frnorm B}{\sqrt{n}}\geq1-t, \\ & \opnorm A \leq 1, \opnorm B \leq1\} \end{align*} where $0<t<1$. So $S_t$ is a compact set in $\mathbb{R}^{n\times n}\times\mathbb{R}^{n\times n}$. Note that as $t\to0$, $S_t\to\{(A,A):A\in \operatorname O(n)\}$.
Define a function $f_{m, n}$ over $\{ (A,B)\in\mathbb{R}^{n\times n}\times\mathbb{R}^{n\times n} : \opnorm A \leq 1, \opnorm B\leq1\}$ as: $$f_{m,n}(A,B)=\det(I-A^{\operatorname T}A)^{(m - (n + 1))/2}\det(I-B^{\operatorname T}B)^{(m - (n + 1))/2}.$$
Can we obtain the following conjecture? $$\frac{\displaystyle\int_{S_{2t}}f_{m,n}(A,B)}{\displaystyle\int_{S_{t}}f_{m,n}(A,B)}\leq C^{mn},$$ where $C$ is a constant independent of $m,n,t$, $m\geq n+1$. The integral is with respect to the Lebesgue measure on $\mathbb{R}^{n\times n}\times\mathbb{R}^{n\times n}$. It seems obvious since the function to be integrated is essentially a polynomial. However, I'm not able to formalize this. The tricky part is the integration region. Is there any possible direction of literature that I can dive into to solve this problem?
This problem has been puzzling me for months. I tried using QR decomposition of $A$ and $B$ as a change of variable, but the term $\Frnorm{A-B}$ is hard to handle. Other terms $\Frnorm A$, $\Frnorm B$, $\opnorm A$, $\opnorm B$ are only related to the spectral property of the matrix, but $\Frnorm{A-B}$ is also related to the orientation of the matrix, which causes the main difficulty.
Some notation definition: $\Frnorm\cdot$ is the Frobenius norm of the matrix, which is the square root of sum of squares of all entries; $\opnorm\cdot$ is the operator norm of the matrix.
