I posted this question on Robotics Stack Exchange (link) but thought it could be relevant here as well.
I'm trying to solve a computer vision problem whereby I wish to use Levenberg–Marquardt non-linear optimization to solve the following equation: $\left(\begin{array}{c}\mathbf{p}_{\min } \\ \mathbf{n}_{\min }\end{array}\right)=\underset{\mathbf{p}, \mathbf{n}}{\operatorname{Argmin}} \underbrace{\sum_{i=1}^{N}\left(I_{k+1}\left(\mathbf{x}_{i}\right)-I_{k}\left(\mathbf{H}_{k+1}(\mathbf{p}, \mathbf{n})\left(\mathbf{x}_{i}\right)\right)\right)^{2}}_{Q(\mathbf{p}, \mathbf{n})}$
Whereby $\mathbf{x}_i$ stands for the $x,y$ coordinates of a provided image, $I$ is a function which represents the grayscale value of the $x,y$ coordinate in the image. For instance, $I(r,c)$ refers to the pixel value at row $r$ and column $c$ in the image, which has fixed dimension e.g $1024 \times 768$. $H$ is the function, or $3\times 3$ homography matrix constructed from $\mathbf{p}$ and $\mathbf{n}$, converting vector $\mathbf{x}_i$ (coordinates $x_i, y_i$) into $\mathbf{x}_i'$ (coordinates $x_i', y_i')$.
Because gradient-related optimization requires the calculation of the Jacobian, can someone tell me how do I formulate the partial deriviatives of the Image function I? I understand that the size of the Jacobian is $n \times 2n$, where $n$ is the number of points to be used in the optimization.
The author of the paper did stated the following, but I did not quite get it.
To numerically compute the optimization, the loss function is further modified by introducing a linearization of the image function $I$ as we do not know directly in which way the image function itself changes when the normal vector $\mathbf{n}$ is modified. Thus, we use the first order term of its Taylor expansion.
