Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

I'm following the open courseware content on Machine Learning from Stanford University. In the lecture notes, it is given that

$$\Delta_A \ tr(ABA^TC) = CAB + C^TAB^T$$

which I tried but couldn't prove easily. It is not required to follow the course content but I just wondered and wanted to learn its proof. Any suggestions?

Update: $A$, $B$, and $C$ are matrices and $\Delta_A$ is the gradient operation on matrix $A$.

share|improve this question
2  
Dear Ismail: it's not necessary to open with a salutation or have a closing like "kind wishes". I guess $A$, $B$, $C$ are square matrices of the same size; could you say what is meant by $\Delta_A$ please? Also, a direct link to the lecture notes would be very nice. –  Todd Trimble Feb 27 '11 at 16:56
1  
Dear Todd: Thanks for you advice. I will take your remarks about how to ask a question into consideration. The course notes are here: stanford.edu/class/cs229/notes/cs229-notes1.pdf Page 9, Eq. 3. –  İsmail Arı Feb 27 '11 at 17:07
2  
I don't see the reason for getting a -1 for the question. Is that because I couldn't prove it easily? If so, why are there 8 current upvotes for the answer post. If I know the reason, I would update my questioning to mathoverflow community accordingly. –  İsmail Arı Apr 10 '11 at 23:23
1  
I didn't downvote the question, but I can see a good reason for doing so, namely, the carelessness with which the question was presented. No indication of what $A$, $B$, $C$, and $\Delta_A$ were, not editing the question to include this information when the defect was brought to your attention - in fact, I've just decided to downvote the question. –  Gerry Myerson Apr 11 '11 at 2:18
    
Thanks for the warning. I updated the question. –  İsmail Arı Apr 11 '11 at 8:27
add comment

1 Answer 1

up vote 10 down vote accepted

I guess $\Delta_A$ denotes the derivative with respect to the elements of the matrix $A$ (more conventionally denoted by $\partial_{A}$).

To evaluate the derivative with respect to $A_{ij}$, write out the trace in terms of components and then use $\partial_{A_{ij}} A_{mn} = \delta_{im} \delta_{jn}$, $$\partial_{A_{ij}} \text{tr}(A B A^T C) = \partial_{A_{ij}}\sum_{mnkl} A_{mn} B_{nk} A_{lk} C_{lm}= \sum_{kl} B_{jk}A_{lk} C_{li} + \sum_{mn} A_{mn} B_{nj} C_{im} $$ $$= ( C^T A B^T+ C A B )_{ij}.$$ This is the component-wise version of your identity.

Note to the comment of Todd Trimble: the matrices $A,B$, and $C$ do not have to be necessarily square matrices. Their dimension just has to "match" ($A \in \mathbb{R}^{m \times n}$, $B\in \mathbb{R}^{n \times n}$, $C \in \mathbb{R}^{m \times m}$, with $m$ and $n$ arbitrary integers).

share|improve this answer
    
Dear Fabian: Your guess is correct. Sorry for missing definition. The operator in question is the gradient operation as you guessed. –  İsmail Arı Feb 27 '11 at 17:14
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.