Let's fix the standard symplectic structure $(\mathbb{R}^{2g}, \omega, J)$. A (marked) symplectic lattice then has the form $A\mathbb{Z}^{2g}$ for $A \in Sp_{2g}\mathbb{R}$. We say a vector subspace $W$ is rational in $\mathbb{Z}^{2g}$ if $W \cap \mathbb{Z}^{2g}$ is a cocompact lattice in $W$, ie. the flat torus $W/W \cap \mathbb{Z}^{2g}$ has finite volume. Likewise we say $W$ is rational in $A \mathbb{Z}^{2g}$ if $A' W$ is rational in $\mathbb{Z}^{2g}$ (where the prime $'$ denotes inverse).

The following problem arises in computations with symplectic lattices: Suppose $W$ is a rational lagrangian subspace in the symplectic lattice $\Lambda$, and suppose we have a basis $w_1, w_2, \ldots$ for $W$ in $\Lambda$, (ie. each $w_i$ is a lattice vector in $\Lambda$). Now for any $A \in Sp_{2g}\mathbb{R}$ how do we compute a basis for $A'W$ in $A\Lambda$?

Some remarks: (we use ${}^o$ and ${}^\perp$ to respectively denote euclidean-orthogonal and $\omega$-orthogonal).

(i) The rationality of $A'W $ in $A \Lambda$ is rather particular to our situation. It is guaranteed by $A$ being symplectic, $W$ being lagrangian, the identity $W^o=JW^\perp=JW$, and the following fact from geometry of numbers: if $W$ is a rational subspace in a lattice $\Lambda$, then $W^o$ is rational in the dual lattice $\Lambda^\ast$ (here 'dual lattice' is meant in the Lekkerkerker or Conway/Sloane sense). Note: the dual $\Lambda^\ast$ of a symplectic lattice $\Lambda$ is given by $J\Lambda$.

(ii) We might not even have $A'W$ be lagrangian in $A\Lambda$, but that's acceptable for us.

(iii) This question arises from trying to understand how the volume of a rational lagrangian subspace grows under linear symplectomorphisms. An alternative version of the above question is as follows: for $w_1, w_2, \ldots$ as basis for $W$ in $\Lambda$ as above, we can compute the volume of $W$ in $\Lambda$ as : $$vol(W, \Lambda)=\det \begin{pmatrix} {}^t w_i w_j\end{pmatrix}.$$ Ie. we take the determinant for a certain symmetric gram matrix. The question that arises as: how do we relate the following two matrices (and in particular, their determinants): $$({}^tw_i w_j),~~~~ ({}^tw_i {}^tA A w_j)?$$

ie. the problem, as before, is comparing $vol(W, \Lambda)$ and $vol(AW, A\Lambda)$.

Note: if we knew a priori that $A$ stabilized $W$ (ie. that $\{Aw_i\}$ was another basis for $W$ in $\Lambda$) then the volume of $AW$ in $A\Lambda$ differs by $det(A|_W)$.

(iv) A more concise statement of the problem is this: let $\mathbb{W}$ denote the matrix whose columns consist of some basis of $W$ in $\Lambda$. Then we are simply looking for a comparison between the determinants of the following matrices: $${}^t\mathbb{W} \mathbb{W}, ~~~{}^t\mathbb{W} {}^tAA\mathbb{W}.$$

Even more concisely, we'd be happy to know how to move ${}^tA$ past ${}^t \mathbb{W}$.

Added: I've been thinking about the question more today and wanted to include some facts on symplectic polar decompositions: let's set $\mathbb{A}={}^tAA$. Then $\mathbb{A}$ is a symmetric positive definite symplectic matrix. Therefore $\mathbb{A}$ is diagonalizable over $\mathbb{R}$ all of whose eigenvalues $\lambda$ are positive real numbers satisfying the following: the eigenvalues occur as quadruples $\{\lambda, \lambda, \lambda^{-1}, \lambda^{-1}\}$ where $|\lambda| \neq 1$. Moreover an eigenspace $V_\lambda$ has $\omega$-orthogonal complement $$V_\lambda^{\perp}=\oplus_{\mu,~\mu \lambda \neq 1} V_\mu.$$

The eigenspace decomposition allows us to rewrite the matrix ${}^t \mathbb{W} \mathbb{A} \mathbb{W}$: if we decompose each $w_i=\Sigma_\lambda w_i^\lambda$ relative to the eigenspaces $\{V_\lambda\}$ of $\mathbb{A}$, then we come to the following expression:

$${}^t \mathbb{W} \mathbb{A} \mathbb{W}=(\Sigma_\lambda \lambda w_i \cdot w_j^\lambda),$$ where $\cdot$ denotes dot-product.

But does anyone have any understanding of how to evaluate the determinant of this matrix in some manner which makes intelligible how it compares to $det(w_i \cdot w_j)$, ie. does the determinant split into eigenvalue components?