Image registration given known gravity direction - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-20T20:58:14Z http://mathoverflow.net/feeds/question/97128 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/97128/image-registration-given-known-gravity-direction Image registration given known gravity direction Alex Flint 2012-05-16T15:08:27Z 2012-05-18T14:54:14Z <p>I am trying to integrate gravity measurements into the estimation of the planar homography relating two images of a planar scene. The cameras are calibrated; that is, a point $X$ in space is transformed into the two images according to </p> <p>$$x_1 = K_1(R_1 X+t_1)$$ $$x_2 = K_2(R_2 X+t_2)$$ where $K_1$ and $K_2$ are known. Without using gravity, the problem looks like</p> <p>$$\min_H d(I_1, I_2, H)$$</p> <p>where $I_1$ and $I_2$ are images and $d(\cdot)$ is some measure of the quality of $H$ as the relationship between them (e.g. pixel-wise SSD).</p> <p>Adding gravity, we measure $g_1$ and $g_2$ in the respective camera coordinate frames and we have $${R_2}^T g_2 = {R_1}^T g_1$$</p> <p>Of course there is noise in the measurements so I would like to incorporate these measurements into the cost function, something like:</p> <p>$$\min_H d(I_1, I_2, H) + h(g_1, g_2, H)$$</p> <p>where $h$ measures the agreement between $H$ and the gravity constraint above. There are a number of reasonable choices for $h$, for example: $$h(g_1, g_2, H) = \| {R_2}^T g_2 - {R_1}^T g_1 \|$$</p> <p>In general I'm not too concerned about the exact form of $h$, so long as it goes to zero as the constraint on $g_1$ and $g_2$ above becomes satisfied. Of course $h$ <em>should</em> be derived from a noise model for the sensor but I'm willing to settle for something "reasonable".</p> <p>I could simply decompose $H$ in the standard way: $$H = K_2 (R - \frac{tn^T}{d}) {K_1}^{-1}$$</p> <p>and this would be fine for <em>evaluating</em> the cost function but not for taking the gradients needed for optimization.</p> <p>Another approach would be to <em>represent</em> $H$ by its rotation and translation component, and then it would be easy to evaluate $h(\cdot)$, but this would break other parts of the system that rely on an existing parametrisation.</p> <p>What I really want is a way to write $h(\cdot)$ as an analytic function of $H$, without assuming any particular parametrisation of $H$.</p>