most recent 30 from http://mathoverflow.net 2013-05-21T01:34:24Z http://mathoverflow.net/feeds/question/11737 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/11737/get-rid-of-tr-in-svm-kernel-trick andinos 2010-01-14T09:13:25Z 2010-01-15T21:29:42Z

<p>I designed a kernel function (to be used within SVM) which has the expression $tr(AB)$ in it. For efficient implementation of this, I was wondering if I could write $tr(AB)$ as an inner product: $\phi(A)^T \phi(B)$? What is the function $\phi()$? </p>

http://mathoverflow.net/questions/11737/get-rid-of-tr-in-svm-kernel-trick/11740#11740 Matus Telgarsky 2010-01-14T09:28:35Z 2010-01-15T10:40:23Z

<p>If $A,B$ are arbitrary $n\times n$ matrices, by definition of trace, $\textrm{tr}(AB) = \sum_{i,j} A_{ij}B_{ji}$. This is $O(n^2)$, but just reading the entries of $A$ is $\Omega(n^2)$. Without any special structure on $A,B$, you probably can't do better.</p> <p>If $A,B$ are (column) vectors, you probably mean the outer product $\textrm{tr}(AB^T) = \sum_i A_i B_i$.</p> <p><strong>Edit</strong>: andinos clarified to say he wants to know about the implicit mapping of the kernel function. Well I have bad news: It does not exist!! The proof works by showing there exist matrices $A,B$ such that the corresponding kernel matrix is not positive semi-definite. To finish, apply Mercer's theorem.</p> <p>In particular, set $A = \left(\begin{array}{cc}1 &amp; 1 \\ -1 &amp; 1\end{array}\right)$ and $B = A^T = \left(\begin{array}{cc}1 &amp; -1 \\ 1 &amp; 1\end{array}\right)$. Therefore $\textrm{tr}(AB) = \textrm{tr}(AA^T) = 4$, and $\textrm{tr}(BA)$ is identical. On the other hand, $\textrm{tr}(AA) = \textrm{tr}(BB) = 0$. therefore, the kernel matrix $K$ is $\left(\begin{array}{cc}0 &amp; 4 \\ 4 &amp; 0\end{array}\right)$. Set $x = \left(\begin{array}{c} 1 \\ -1\end{array}\right)$, and observe that $x^T K x = -8 &lt; 0$, and therefore $K$ is not PSD, so the kernel $k(A,B) = \textrm{tr}(AB)$ is not PSD.</p> <p>On the other hand! If you had instead defined your kernel to be $k'(A,B) = \textrm{tr}(AB^T)$, notice that $k'(A,B) = \sum_{i,j}A_{ij}B_{ij} = \Phi(A)^T\Phi(B)$ where $\Phi$ simply takes its input matrix and outputs it as a column vector.</p>

http://mathoverflow.net/questions/11737/get-rid-of-tr-in-svm-kernel-trick/11924#11924 deneme 2010-01-15T21:29:42Z 2010-01-15T21:29:42Z

<p>Matus is right. But if the matrices $A$, and $B$ have certain properties like being symmetric, or diagonal, then simply just vectorizing the matrices and taking their inner product would be equal to the $tr(AB)$. </p>