The best unitary matrices that approximate a matrix product

Question

Let $\mathbf{A}$ be an arbitrary $N\times N$ complex matrix. Moreover, $\mathcal{U}_1$ and $\mathcal{U}_2$ are distinct subsets of all unitary matrices. Suppose the matrices $\mathbf{U}_1$ and $\mathbf{V}_1$ are selected from $\mathcal{U}_1$. What is the solution of the following optimization problem \begin{align} \min_{\mathbf{U}_2,\mathbf{V}_2\in\mathcal{U}_2} \|\mathbf{U}_2\mathbf{A}\mathbf{V}_2-\mathbf{U}_1\mathbf{A}\mathbf{V}_1\|_F \end{align} Can we say that the optimal $\mathbf{U}_2$ and $\mathbf{V}_2$ are the solution of the following \begin{align} \mathbf{U}_2^\ast&=\text{arg}\min_{\mathbf{U}_2\in\mathcal{U}_2} \|\mathbf{U}_2-\mathbf{U}_1\|_F\\ \mathbf{V}_2^\ast&=\text{arg}\min_{\mathbf{V}_2\in\mathcal{U}_2} \|\mathbf{V}_2-\mathbf{V}_1\|_F \end{align}