4
$\begingroup$

Hi,
I am optimizing a function over a matrix $U$, where $U \in \mathbb{R}^{m \times n}$ and $U^TU = I$. I do not want to run a constrained maximization program, since employing the constraint $U^TU = 1$ with lagrange multipliers would lead to higher time for convergence. I instead want to run simple gradient based optimization. So, I wanted to encode the orthogonality constraint into the input variable itself. What I mean by that is, for example, if $U$ was a square matrix then by Cayley transform for a skew-symmetric matrix $A$ we have: $U = (I-A)(I+A)^{-1}$ -- and then I can use unconstrained optimization with a lower dimensionality of input variables.

So my question is whether there is an extension of the Cayley transform to non-square matrices of type $U^TU = I$ where $U \in \mathbb{R}^{m \times n}$.

Thanks
I.J

$\endgroup$

3 Answers 3

4
$\begingroup$

Have a look at the following slides (several pointers are in there)

Optimization on the Stiefel manifold

The point is that you can directly remain on the manifold while optimizing, so no explicit "constraint enforcement" will be required.

$\endgroup$
0
4
$\begingroup$

My Friend, did you try the following: Fix a feasible matriz Y (Y^T*Y = I). So you can parametrize your subset by

Y = (I+A)^{-1}*(I-A)*Y_0,

where A, skew-symmetric, is now the variables.

$\endgroup$
1
  • $\begingroup$ Hi Juliano, I used conjugate gradient on Stiefel manifolds. Your suggestion of writing any point on the Stiefel manifold as product of m x m unitary matrix with a fixed m x n Y_0 is very neat. cheers $\endgroup$
    – I J
    Aug 23, 2011 at 5:14
2
$\begingroup$

Now there is a new paper about this, may be it would be useful: Cayley Transform on Stiefel manifolds, Enrique Macías-Virgós, María José Pereira-Sáez, Daniel Tanré. arXiv:1612.07142

$\endgroup$
1
  • $\begingroup$ That paper looks extremely applicable, but it is a little hard to approach. Is there any way you can help distill their result? In the Cayley Transform you parameterize with a skew-symmetric matrix and the function that produces the square orthogonal matrix is straightforward - but I'm not sure what that function looks like from this paper :( $\endgroup$
    – Robert
    Apr 10, 2018 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.