MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am optimizing a function over a matrix $U$, where $U \in \mathbb{R}^{m \times n}$ and $U^TU = I$. I do not want to run a constrained maximization program, since employing the constraint $U^TU = 1$ with lagrange multipliers would lead to higher time for convergence. I instead want to run simple gradient based optimization. So, I wanted to encode the orthogonality constraint into the input variable itself. What I mean by that is, for example, if $U$ was a square matrix then by Cayley transform for a skew-symmetric matrix $A$ we have: $U = (I-A)(I+A)^{-1}$ -- and then I can use unconstrained optimization with a lower dimensionality of input variables.

So my question is whether there is an extension of the Cayley transform to non-square matrices of type $U^TU = I$ where $U \in \mathbb{R}^{m \times n}$.


share|cite|improve this question
up vote 3 down vote accepted

Have a look at the following slides (several pointers are in there)

Optimization on the Stiefel manifold

The point is that you can directly remain on the manifold while optimizing, so no explicit "constraint enforcement" will be required.

share|cite|improve this answer

My Friend, did you try the following: Fix a feasible matriz Y (Y^T*Y = I). So you can parametrize your subset by

Y = (I+A)^{-1}*(I-A)*Y_0,

where A, skew-symmetric, is now the variables.

share|cite|improve this answer
Hi Juliano, I used conjugate gradient on Stiefel manifolds. Your suggestion of writing any point on the Stiefel manifold as product of m x m unitary matrix with a fixed m x n Y_0 is very neat. cheers – I J Aug 23 '11 at 5:14

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.