MathOverflow will be down for maintenance for approximately 3 hours, starting Monday evening (06/24/2013) at approximately 9:00 PM Eastern time (UTC-4).
show/hide this revision's text 2 Replaced max by \max.

Your problem is in the form

$max_x \max_x f(x)$ subject to $g(x) < 0$, where $x$ is the set of components $x_i^j$ of the matrix $X$.

Use one Lagrange multiplier to write $f(x) + \lambda g(x)$. Both nonlinear terms are quadratic, so this is the most favorable case for the purpose of obtaining solutions. To take the derivative of the expression above with respect to the set of unknowns $x$, index notation as in tensor calculus could be helpful. Then you are in the standard form of constrained optimization.

show/hide this revision's text 1

Your problem is in the form

$max_x f(x)$ subject to $g(x) < 0$, where $x$ is the set of components $x_i^j$ of the matrix $X$.

Use one Lagrange multiplier to write $f(x) + \lambda g(x)$. Both nonlinear terms are quadratic, so this is the most favorable case for the purpose of obtaining solutions. To take the derivative of the expression above with respect to the set of unknowns $x$, index notation as in tensor calculus could be helpful. Then you are in the standard form of constrained optimization.