Rational orthogonal matrices ``everybody knows'' that an integral orthogonal matrix is a signed permutation matrix, so there are exactly $2^n n!$ such matrices in $O(n).$ Now, what if we ask for the enumeration of elements of $O(n$ with rational elements, where all the denominators are equal to $q$ (where $q$ is an arbitrary integer, though if the question is much easier for $q$ prime, that's fine too...)? I would suspect this has been studied... (one can ask the same question for arbitrary algebraic groups/number fields, of course).
 A: Boris Venkov asked this question, once, in the coffea room of the Maths Department of the university of Geneva. 
Here are some things that relate to the case $q=2^m$.
Let $I_n$ be the 'euclidean' $\mathbf Z$-module of rank $n$, and $A\mapsto G_n(A)$ the group it defines.
Let us write $a(n,m)$ for the number of matrices in $G_n(\mathbf Z[\frac{1}{2}])$ all of whose coordinates have $2$-valuation at least $-m$. 
First note that $G_n(\mathbf Z[\frac{1}{2}])$ is finite for $n<5$.
In fact, one even has $G_n(\mathbf Z[\frac{1}{2}])=G_n(\mathbf Z)\simeq\mathbf Z/2^n\rtimes S_n$ for $n<4$ and $G_4(\mathbf Z[\frac{1}{2}])=Aut(D_4)=G_4(\mathbf Z)\rtimes \mathbf Z/3$, where $D_4$ denotes the sublattice of elements of even length in $I_4$. The $\mathbf Z/3$ is generated by 
$$ \begin{bmatrix}
-1/2 & 1/2 & 1/2 & 1/2\\\
-1/2 & 1/2 & -1/2 & -1/2\\\
-1/2 & -1/2 & 1/2 & -1/2\\\
-1/2 & -1/2 & -1/2 & 1/2
\end{bmatrix}$$
Thus in this case, you have $a(4,0)=384=2^7*3$ and $a(4,1)=1152=2^7*3^2$.
Next, note that $G_5(\mathbf Z[\frac{1}{2}])$ is an amalgamated sum : $A\star_C B$ with $A=G_5(\mathbf Z)$, $B=Aut(I_1)\times Aut(D_4)$ and $C=A\cap B\simeq \mathbf Z/2\times G_4(\mathbf Z)$. 
This decomposition comes from the action of $G_5(\mathbf Z[\frac{1}{2}])$ on the Bruhat-Tits Building of $G_5(\mathbf Q_2)$ : a tree whose vertices are 5-valent and 3-valent.
The number $\frac{a(5,m)}{\vert G_5(\mathbf Z)\vert}$ counts the $5$-valent vertices that are at (combinatorial) distance less than $2m$ from a fixed one. Thus 
$\frac{a(5,m)}{\vert G_5(\mathbf Z)\vert}=10*\frac{8^m-1}{7}+1$.
Things are becoming more complicated in higher dimensions since the Bruhat-Tits building is not a tree anymore ($n\geq 6$) and the action of $G_n(\mathbf Z[\frac{1}{2}])$ on the vertices of a same type (say corresponding to unimodular lattices) is not transitive anymore ($n\geq 9$).
You may adapt the same method to obtain similar results for small values of $n$ and $q=p^m$ : the Bruhat-Tits building is a tree for $n=3$, the action of $G_3(\mathbf Z[\frac{1}{p}])$ is transitive on one type of vertices (but rarely on maximal simplices) ... 
A: This is not a complete answer, but it's a start.
In the $2 \times 2$ case, the act of choosing the first column and clearing denominators describes a two-to-one map from orthogonal matrices to primitive vectors in the lattice $\mathbb{Z} \oplus \mathbb{Z}$.  In particular, there are twice as many orthogonal matrices with denominator exactly $q^2$ as there are primitive vectors of length $q$, and there are twice as many orthogonal matrices with denominator dividing $q^2$ as there are vectors of length $q$.  The latter quantity is enumerated by the theta function of the lattice, which is a modular form of weight 1, and its coefficients grow logarithmically.
In higher dimension, I guess the generating function comes from successively choosing vectors in orthogonal complements, so it should be at least related to modular forms.
