Rotation matrix between two column spaces

Question

I would like to find a rotation matrix between two flats $F_1,F_2$ that are defined by the column spaces of the matrices $M_1,M_2 \in \mathbb{R}^{n \times k}$ ($k<n$) respectively. If it was to find any rotation matrix, I could do the SVD (Singular Value Decomposition) or Gram-Schmidt process to find the orthonormal bases $U_1$ of $M_1$ and $U_2$ of $M_2$, and define rotation matrix $R=U_2 U_1^{\top}$. But what I need is to find the rotation along the line of intersection $L$.

Specifically, if $k=2$ and $n=3$, the flats would look something like this figure. I have a vector $v_1$ lies on $F_1$, which enters intersection $L$. As it passes through $L$, $v_1$ is transformed to $v_2$ such that it lies on $F_2$. This should not be confused with a projection (My interest is related to finding the shortest path from point on $F_1$ to $F_2$.).

Do you have any idea on this problem? One idea I come up with is:

1. First, find the intersection $L$ by noting that $L$ is a set of points satisfies $P=M_1 \vec{a_1}=M_2 \vec{a_2}$. i.e., $[M_1, M_2][\vec{a_1}; -\vec{a_2}]=0$, thus $L$ is the null space of $[M_1, M_2]$ ($[\cdot,\cdot]$ and $[\cdot;\cdot]$ are horizontal and vertical concatenation respectively).
2. Find the angle between flats by taking non-zero, non-one singular values of $\bar{U_1}^\top \bar{U_2}$, where $\bar{U}_\cdot\in\mathbb{R}^{n\times k}$ is a thin singular vector matrix (set of singular vectors that have non-zero singular values).
3. Construct rotation matrix using axes and anlge obtained from the previous steps.

However, this method is too cumbersome. I think there is more compact and closed-form solution.

Answer 1

Here is a closed-form answer for the generic case with $k=2, n=3$ that makes me doubt there will be a compact answer in general.

Let $p,q$ be vectors perpendicular to $F_1, F_2$ respectively. Let $r = p \times q$, where $\times$ is the cross product, so that $r$ is in the intersection of $F_1$ and $ F_2$. Then the desired map is

$$v \rightarrow \frac{(v\cdot p)q}{|p||q|} + \frac{(v\cdot r)r}{|r||r|} + \frac{(v\cdot (p\times r))(q\times r)}{|p\times r||q\times r|}. $$

This can be verified by checking that it sends the orthogonal basis $p,r,p\times r$ to the orthogonal basis $q,r,q\times r$, up to some scalar multiples chosen to preserve lengths.