Let's assume a camera in space and an image point in this camera:
$t \in \mathbb{R}^3$ is the position of the camera in space.
$R \in \mathbb{R}^{3 \times 3}$ is the orientation of the camera in space.
$K \in \mathbb{R}^{3 \times 3}$ is the camera intrinsic matrix.
$M = (R^T|-R^Tt) \mathbb{R}^{3 \times 4}$ is the camera extrinsic matrix.
The
- $t \in \mathbb{R}^3$ is the position of the camera in space.
- $R \in \mathbb{R}^{3 \times 3}$ is the orientation of the camera in space.
- $K \in \mathbb{R}^{3 \times 3}$ is the camera intrinsic matrix.
- $M = (R^T|-R^Tt) \mathbb{R}^{3 \times 4}$ is the camera extrinsic matrix.
The projection in homogenous coordinates between the camera and real world is:
$\begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = K M\begin{bmatrix} X \\ Y \\ Z \\ 1 \end{bmatrix}$$$\begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = K M\begin{bmatrix} X \\ Y \\ Z \\ 1 \end{bmatrix}$$
Consider a plane $E = \{x \in \mathbb{R}^3| x^T n=0 \}$ with normal vector $n \in \mathbb{R}^3$.
Let's consider the case where the ray $L = \{\lambda\begin{bmatrix} X \\ Y \\ Z \end{bmatrix} | \lambda \in \mathbb{R}^3 \}$ has
$$
L = \left\{\lambda\begin{bmatrix} X \\ Y \\ Z \end{bmatrix}\; : \; \lambda \in \mathbb{R}^3 \right\}
$$
has a unique intersection with $E$, the surface point $S \in E \cap V$.
Moreover let's put
Let's say $R = R_1(\varphi_{roll}) R_2(\varphi_{pitch}) R_3(\varphi_{yaw})$.
$$
R = R_1(\varphi_\text{roll}) R_2(\varphi_\text{pitch}) R_3(\varphi_\text{yaw}).
$$
Now we assume angular errorslet's consider the $\phi_{roll}, \phi_{pitch}, \phi_{yaw} \sim N(0, \sigma_{roll}), N(0, \sigma_{pitch}), N(0, \sigma_{yaw})$.
And spacial errorsangular errors $T_x, T_y, T_z \sim N(0, \sigma_x), N(0, \sigma_y), N(0, \sigma_z)$.
Then$\phi_\text{roll}, \phi_\text{pitch}, \phi_\text{yaw}$ respectively distributed as $R_{error} = R R_1(\phi_{roll}) R_2(\phi_{pitch}) R_3(\phi_{yaw})$$N(0, \sigma_\text{roll}), N(0, \sigma_\text{pitch}), N(0, \sigma_\text{yaw})$, and the spatial errors $t_{error} = t + \begin{bmatrix} T_x \\ T_y \\ T_z \end{bmatrix}$$T_x, T_y, T_z$ respectively distributed as $N(0, \sigma_x), N(0, \sigma_y), N(0, \sigma_z)$: then
$$
R_\text{error} = R\cdot R_1(\phi_\text{roll}) R_2(\phi_\text{pitch}) R_3(\phi_\text{yaw})
$$
and
$$
t_\text{error} = t + \begin{bmatrix} T_x \\ T_y \\ T_z \end{bmatrix}
$$
Without loss of generality we may assume $E = \mathbb{R}^2 \times \{0\}$ and we can identify $S \in \mathbb{R}^3$ as $S' \in \mathbb{R}^2 $.
I am interested in an approximate computation of the covariance of $\Sigma_{S'}$, which is the uncertainty that results in the projection of the image point to the surface point $S'$, which also becomes a random variable if the position and angles of the observing camera are random. We can assume that all angular errors are small and approximate Taylor series can be used for our purposes.
