I should say **yes**. For this, I shall use the fact that in the unit sphere $\mathbb S^{d-1}$, the set of rational vectors is dense. I shall proceed by induction over $n$.

So let $A\in {\bf O}_n(\mathbb R)$ be given. Let $\vec v_1$ be its first column, an element of ${\mathbb S}^{n-1}$. We can choose a rational unit vector $\vec w_1$ arbitrarily close to $\vec v_1$. The first step is to construct a rational orthogonal matrix $B$ with first column $\vec w_1$. To this end we choose inductively rational unit vectors $\vec w_2,\ldots,\vec w_n$. This is possible because at each step, we may take a rational unit vector in the unit sphere of a "rational" subspace. Here, a subspace $F$ is rational if it admits a rational basis.

Now, let us form $A_1=B^{-1}A$. This is a orthogonal matrix, whose first column is arbitrarily close to $\vec e_1$. Hence its first line is close to $(1,0,\ldots,0)$ as well. Thus
$$A_1\sim\begin{pmatrix} 1 & 0^T \\\\ 0 & R \end{pmatrix}.$$
The matrix $R$ is arbitrarily close to ${\bf O}_{n-1}({\mathbb R})$. By the induction hypothesis, there exists a rational orthogonal matrix $Q$ arbitraly close to $R$. Then
$$B\begin{pmatrix} 1 & 0^T \\\\ 0 & Q \end{pmatrix}$$
is arbitrarily close to $A$.