MathOverflow will be down for maintenance for approximately 3 hours, starting Monday evening (06/24/2013) at approximately 9:00 PM Eastern time (UTC-4).
show/hide this revision's text 2 edited tags
show/hide this revision's text 1

How do I optimize over (or take derivative wrt) a square diagonal matrix?

Hello. I'd like to solve the following optimization problem.

$P_i$ is a 6x6 matrix
$X$, $Y$ is a 6xk matrix
$w_i$ is a kx1 vector
$diag(w_i)$ is a square diagonal matrix with diagonal entries equal to $w_i$

$\min_{w_i} ~ ||P_i - X diag(w_i) Y^T||_F^2$

So the question is how to optimize over $diag(w_i)$.

Does anyone know how to take derivative wrt a diagonal matrix?

Or would it work if treat $diag(w_i)$ as a square matrix, solve it,
and then set off-diagonal entries to zeros?