Question

Hi,
I am optimizing a function over a matrix $U$, where $U \in \mathbb{R}^{m \times n}$ and $U^TU = I$. I do not want to run a constrained maximization program, since employing the constraint $U^TU = 1$ with lagrange multipliers would lead to higher time for convergence. I instead want to run simple gradient based optimization. So, I wanted to encode the orthogonality constraint into the input variable itself. What I mean by that is, for example, if $U$ was a square matrix then by Cayley transform for a skew-symmetric matrix $A$ we have: $U = (I-A)(I+A)^{-1}$ -- and then I can use unconstrained optimization with a lower dimensionality of input variables.

So my question is whether there is an extension of the Cayley transform to non-square matrices of type $U^TU = I$ where $U \in \mathbb{R}^{m \times n}$.

Thanks
I.J

Answer 1

Have a look at the following slides (several pointers are in there)

Optimization on the Stiefel manifold

The point is that you can directly remain on the manifold while optimizing, so no explicit "constraint enforcement" will be required.

Answer 2

My Friend, did you try the following: Fix a feasible matriz Y (Y^T*Y = I). So you can parametrize your subset by

Y = (I+A)^{-1}*(I-A)*Y_0,

where A, skew-symmetric, is now the variables.