# Is there a relation between the Hessian matrix and the structure Tensor?

for a 2 dimensional image, i am interested to find a relation between the Hessian $H = \begin{pmatrix} \frac{\partial^2 I}{\partial x^2}&\frac{\partial^2 I}{\partial x \partial y}\\ \frac{\partial^2 I}{\partial x \partial y}&\frac{\partial^2 I}{\partial y^2} \end{pmatrix}$ and the structure tensor $ST = \begin{pmatrix} w*\frac{\partial I}{\partial x}\frac{\partial I}{\partial x}&w*\frac{ \partial I}{\partial x}\frac{ \partial I}{\partial y}\\ w*\frac{\partial I}{\partial x}\frac{ \partial I}{\partial y}&w*\frac{\partial I}{\partial y}\frac{ \partial I}{\partial y}, \end{pmatrix}$, where $w$ is a weight function. I am using a Gaussiankernel. As far as i know, the structure tensor is also called "autocorrelation matrix" or "second moment matrix".

I am using these two matrices to find so called "interesting points" in an image. For the $ST$ i can find the eigenvectors $v_1$ und $v_2$ with the corresponding eigenvalues $\lambda_1>\lambda_2$.

$v_1$ should have a orientation with the highest grayvalues fluctuation, so i think you can say it is a normalvector. $v_2$ stands orthogonal to $v_1$ and should have a orientation with the lowest grayvalues fluctuation.

I found out that the eigenvalues of the Hessianmatrix can be seen as the principal curvatures $\kappa_1$ $\kappa_2$. Because for both matrices the determinant of the matrix gives us $det(M)=\lambda_1 \cdot \lambda_2$, i can calculate for the hessian $det(H)=\lambda_1 \cdot \lambda_2=\kappa_1 \cdot \kappa_2$. The eigenvectors of the Hessian gives us the principal directions.

If i want to find a interesting point, i calculate for all points in the image, the determinante of both matrices. If i say that the determinant should be max i find with both matrices some interesting point. This means that both eigenvectors have a large magnitude and the point should be a corner in the image or any other singularity. Generally i observed that the ST gives me fewer points than the Hessian and that most of the points detected with ST is also found with the Hessian (the interesting points dont have exactly same position, but mostly the are only one pixel away. so i take it as the same points).

So what i want to know is: If i take the ST can i transform it somehow and get the Hessianmatrix? Because the eigenvalues of both matrices are important to me,i want to know if i can find the eigenvalues or an approximation of the Hessian using the ST. I am also interested to find a reverse way for this problem. I found an approach which i think could help but i cant make anything out of this. The Weingarten mapping matrix is $W = M \cdot T$ with.

$M = \begin{pmatrix} 1+\frac{\partial I}{\partial x}\frac{\partial I}{\partial x}&\frac{ \partial I}{\partial x}\frac{ \partial I}{\partial y}\\ \frac{\partial I}{\partial x}\frac{ \partial I}{\partial y}&1+\frac{\partial I}{\partial y}\frac{ \partial I}{\partial y}, \end{pmatrix}$ and

$T = (1+\frac{ \partial I}{\partial x}\frac{ \partial I}{\partial x}+\frac{ \partial I}{\partial x}\frac{ \partial I}{\partial x})^{-1/2} \cdot H$

thank you.

• The definition of $ST$ is unclear to me. I presume that there is a convolution product, but parentheses are laking. Do you mean $w*(f_xf_x)$, or $(w*f_x)f_x$ ? – Denis Serre May 24 '18 at 9:17