Proof of a fact about traces - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-22T21:54:53Z http://mathoverflow.net/feeds/question/56836 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/56836/proof-of-a-fact-about-traces Proof of a fact about traces İsmail Arı 2011-02-27T16:40:13Z 2011-04-11T08:35:12Z <p>I'm following the open courseware content on Machine Learning from Stanford University. In the <a href="http://www.stanford.edu/class/cs229/notes/cs229-notes1.pdf" rel="nofollow">lecture notes</a>, it is given that</p> <p>$$\Delta_A \ tr(ABA^TC) = CAB + C^TAB^T$$</p> <p>which I tried but couldn't prove easily. It is not required to follow the course content but I just wondered and wanted to learn its proof. Any suggestions?</p> <p>Update: $A$, $B$, and $C$ are matrices and $\Delta_A$ is the gradient operation on matrix $A$. </p> http://mathoverflow.net/questions/56836/proof-of-a-fact-about-traces/56838#56838 Answer by Fabian for Proof of a fact about traces Fabian 2011-02-27T17:00:20Z 2011-02-27T17:05:58Z <p>I guess $\Delta_A$ denotes the derivative with respect to the elements of the matrix $A$ (more conventionally denoted by $\partial_{A}$).</p> <p>To evaluate the derivative with respect to $A_{ij}$, write out the trace in terms of components and then use $\partial_{A_{ij}} A_{mn} = \delta_{im} \delta_{jn}$, $$\partial_{A_{ij}} \text{tr}(A B A^T C) = \partial_{A_{ij}}\sum_{mnkl} A_{mn} B_{nk} A_{lk} C_{lm}= \sum_{kl} B_{jk}A_{lk} C_{li} + \sum_{mn} A_{mn} B_{nj} C_{im}$$ $$= ( C^T A B^T+ C A B )_{ij}.$$ This is the component-wise version of your identity.</p> <p>Note to the comment of Todd Trimble: the matrices $A,B$, and $C$ do not have to be necessarily square matrices. Their dimension just has to "match" ($A \in \mathbb{R}^{m \times n}$, $B\in \mathbb{R}^{n \times n}$, $C \in \mathbb{R}^{m \times m}$, with $m$ and $n$ arbitrary integers).</p>