I would like to find a rotation matrix between two flats $F_1,F_2$ that are defined by the column spaces of the matrices $M_1,M_2 \in \mathbb{R}^{n \times k}$ ($k<n$) respectively. If it was to find any rotation matrix, I could do the SVD (Singular Value Decomposition) or Gram-Schmidt process to find the orthonormal bases $U_1$ of $M_1$ and $U_2$ of $M_2$, and define rotation matrix $R=U_2 U_1^{\top}$. But what I need is to find the rotation along the line of intersection $L$.

Specifically, if $k=2$ and $n=3$, the flats would look something like this figure. I have a vector $v_1$ lies on $F_1$, which enters intersection $L$. As it passes through $L$, $v_1$ is transformed to $v_2$ such that it lies on $F_2$. This should not be confused with a projection (My interest is related to finding the shortest path from point on $F_1$ to $F_2$.).

Do you have any idea on this problem? One idea I come up with is:

  1. First, find the intersection $L$ by noting that $L$ is a set of points satisfies $P=M_1 \vec{a_1}=M_2 \vec{a_2}$. i.e., $[M_1, M_2][\vec{a_1}; -\vec{a_2}]=0$, thus $L$ is the null space of $[M_1, M_2]$ ($[\cdot,\cdot]$ and $[\cdot;\cdot]$ are horizontal and vertical concatenation respectively).
  2. Find the angle between flats by taking non-zero, non-one singular values of $\bar{U_1}^\top \bar{U_2}$, where $\bar{U}_\cdot\in\mathbb{R}^{n\times k}$ is a thin singular vector matrix (set of singular vectors that have non-zero singular values).
  3. Construct rotation matrix using axes and anlge obtained from the previous steps.

However, this method is too cumbersome. I think there is more compact and closed-form solution.

  • 4
    $\begingroup$ Shorter version of everything after the first sentence: "How can I find a rotation matrix U such that UF_1=F_2 and U is the identity on F_1 intersect F_2?" $\endgroup$
    – user44143
    Commented Apr 20, 2017 at 7:43
  • 1
    $\begingroup$ @MattF. Thank you for a succinct summary. $\endgroup$
    – nzer0
    Commented Apr 20, 2017 at 10:59

1 Answer 1


Here is a closed-form answer for the generic case with $k=2, n=3$ that makes me doubt there will be a compact answer in general.

Let $p,q$ be vectors perpendicular to $F_1, F_2$ respectively. Let $r = p \times q$, where $\times$ is the cross product, so that $r$ is in the intersection of $F_1$ and $ F_2$. Then the desired map is

$$v \rightarrow \frac{(v\cdot p)q}{|p||q|} + \frac{(v\cdot r)r}{|r||r|} + \frac{(v\cdot (p\times r))(q\times r)}{|p\times r||q\times r|}. $$

This can be verified by checking that it sends the orthogonal basis $p,r,p\times r$ to the orthogonal basis $q,r,q\times r$, up to some scalar multiples chosen to preserve lengths.

  • $\begingroup$ Thank you for the answer. I guess the first term is not necessary? because $v \cdot p$ is always zero as $ v \in F_1$. Anyways, I think I should wait for other answers because I really need the solution for general $k$ and $n$. $\endgroup$
    – nzer0
    Commented Apr 24, 2017 at 0:47
  • $\begingroup$ @nzer0, since you've waited over two and a half years, it's probably appropriate to accept this. $\endgroup$
    – LSpice
    Commented Jan 21, 2020 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.