I wrote an earlier post with the same question, but was asked to close the post. I now try again, essentially by adding a little background and pointing out a connection to an already studied problem,
Wahba's problem is the following:
$$\min_R \sum_{k=1}^K \|v_k - Rw_k\|^2$$ where $v_k$ and $w_k$ are arbitrary $3\times 1$ vectors, and $R$ is a rotation matrix (i.e., orthogonal with $\det(R)=1$). A variant of Wahba's problem is the orthogonal Procrustes problem.
I have a more complicated variant of Wahba's problem.
- I only study $K=3$.
- I have $w_k=R_{u_k}(\alpha_k)v_k$ where $R_{u_k}(\alpha_k)$ is a rotation around the axis $u_k$ with an angle $\alpha_k$.
- The axes $u_k$ are given in advance.
- I can optimize over $\alpha_k$.
Thus, my variant of Wahba's read $$\min_{R, \alpha_k} \sum_{k=1}^K \|v_k - RR_{u_k}(\alpha_k)v_k\|^2$$ where $v_k$ and $u_k$ are given. My claim is that there exist non-trivial solutions, i.e., $R\neq I$ and $\alpha_k\neq 0$ so that the minimum value above is 0. Note that for $R=I, \alpha_k=0$, we trivially obtain a minumum value of 0. To be more precise, what I observed in matlab is that if I choose $v_k$ and $u_k$ randomly, I always have multiple solutions.
The background of this problem is that the orientation of a device cannot be uniquely determined using three measurements of incoming signal directions if said measurements are done with linear arrays; linear arrays result in ambiguity of incoming direction (the $\alpha_k$s). What I try to say is that this is not a homework problem, but industrial research with a practical application.