Is there a relation between the Hessian matrix and the structure Tensor?

Question

for a 2 dimensional image, i am interested to find a relation between the Hessian $H = \begin{pmatrix} \frac{\partial^2 I}{\partial x^2}&\frac{\partial^2 I}{\partial x \partial y}\\ \frac{\partial^2 I}{\partial x \partial y}&\frac{\partial^2 I}{\partial y^2} \end{pmatrix}$ and the structure tensor $ST = \begin{pmatrix} w*\frac{\partial I}{\partial x}\frac{\partial I}{\partial x}&w*\frac{ \partial I}{\partial x}\frac{ \partial I}{\partial y}\\ w*\frac{\partial I}{\partial x}\frac{ \partial I}{\partial y}&w*\frac{\partial I}{\partial y}\frac{ \partial I}{\partial y}, \end{pmatrix}$, where $w$ is a weight function. I am using a Gaussiankernel. As far as i know, the structure tensor is also called "autocorrelation matrix" or "second moment matrix".

I am using these two matrices to find so called "interesting points" in an image. For the $ST$ i can find the eigenvectors $v_1$ und $v_2$ with the corresponding eigenvalues $\lambda_1>\lambda_2$.

$v_1$ should have a orientation with the highest grayvalues fluctuation, so i think you can say it is a normalvector. $v_2$ stands orthogonal to $v_1$ and should have a orientation with the lowest grayvalues fluctuation.

I found out that the eigenvalues of the Hessianmatrix can be seen as the principal curvatures $\kappa_1$ $\kappa_2$. Because for both matrices the determinant of the matrix gives us $det(M)=\lambda_1 \cdot \lambda_2$, i can calculate for the hessian $det(H)=\lambda_1 \cdot \lambda_2=\kappa_1 \cdot \kappa_2$. The eigenvectors of the Hessian gives us the principal directions.

If i want to find a interesting point, i calculate for all points in the image, the determinante of both matrices. If i say that the determinant should be max i find with both matrices some interesting point. This means that both eigenvectors have a large magnitude and the point should be a corner in the image or any other singularity. Generally i observed that the ST gives me fewer points than the Hessian and that most of the points detected with ST is also found with the Hessian (the interesting points dont have exactly same position, but mostly the are only one pixel away. so i take it as the same points).

So what i want to know is: If i take the ST can i transform it somehow and get the Hessianmatrix? Because the eigenvalues of both matrices are important to me,i want to know if i can find the eigenvalues or an approximation of the Hessian using the ST. I am also interested to find a reverse way for this problem. I found an approach which i think could help but i cant make anything out of this. The Weingarten mapping matrix is $W = M \cdot T$ with.

$M = \begin{pmatrix} 1+\frac{\partial I}{\partial x}\frac{\partial I}{\partial x}&\frac{ \partial I}{\partial x}\frac{ \partial I}{\partial y}\\ \frac{\partial I}{\partial x}\frac{ \partial I}{\partial y}&1+\frac{\partial I}{\partial y}\frac{ \partial I}{\partial y}, \end{pmatrix}$ and

$T = (1+\frac{ \partial I}{\partial x}\frac{ \partial I}{\partial x}+\frac{ \partial I}{\partial x}\frac{ \partial I}{\partial x})^{-1/2} \cdot H $

thank you.

Answer 1

The ST can be related to the hessian of the first order taylor expansion of a non-linear function you evaluate the hessian on. There is a lot of information on this subject from people performing gradient descent algorithm as the outer product of the jacobian (ST for an image) is a one of the quasi-Newton methods known as Gauss-Newton method.

While the gradient contains information on the orientation, the structure tensor is a way to have an average of the orientation that respect orthogonality as it is needed to compute eigen{values|vectors}. This helps to find corner by analyzing invariant of this tensor, which contain only orientation information. So if you have two high eigenvalues you are at a corner, one you are at an edge.

The hessian matrix would tend to be infinite on the edge and corner as the curvature is infinite ... So indeed it could be interesting for corner detection. You have to keep in mind that the impact of the noise on derivation (derivation ca be seen as a high pass filter) is very important so when you have a filter with sufficient characteristic length for the structure tensor, it won't be necessarily the case for the hessian matrix, so you will have to increase your characteristic length ...

It is not obvious to state if it is better to increase your characteristic length or to approximate your hessian matrix for detection purposes.