My overdetermined linear system gives both bad and good estimates. Why ?

Question

Hello to everyone.

What the question means is that different ways of expressing the same relation between the data and unknown variables produce really weird fit results:

The problem: I have the unknown variables $x_1,x_2,x_3,x_4,x_5,x_6$ and the observations $\vec{q}_i=\Big(q_i(1),q_i(2),q_i(3)\Big), i=1\ldots N$
($N$ observations of real valued 3-vectors)

The relation between the unknowns and the observations is:

$\vec{q}_i\cdot(x_1,x_3,x_5)^T=-1$ (1st Set of $N$ equations)
$\vec{q}_i\cdot(x_2,x_4,x_6)^T=3$ (2nd Set of $N$ equations)

If i solve each set of equations as a separate overdetermined linear system the solution error is very bad.

But if i exploit that $3=(-3)*(-1)$ and combine the 2 previous Equations into one equation

$\vec{q}_i\cdot(x_2,x_4,x_6)^T=-3\ \vec{q}_i\cdot(x_1,x_3,x_5)^T$

This is a relation between the observations and the unknowns that results in the following overdetermined homogeneous system:

$3q_i(1)x_1+q_i(1)x_2+3q_i(2)x_3+q_i(2)x_4+3q_i(3)x_5+q_i(3)x_6=0,\ \ \ \ \ i=1\ldots N$

I use MATLAB for calculations.
Solving this for $x_1,x_2,x_3,x_4,x_5,x_6$ using SVD gives excellent fitting results and extremely low error.
What is so bad about solving separately the two overdetermined linear systems?
Are my variables correlated?
What are some guidelines to be followed when formulating overdetermined systems of equations?

I have been frantically looking for answer on this one for days.

EDIT: The homogeneous representation is the only one that is correct. It turns out that the inhomogeneous representation using constants on the right side for the equations is wrong.

In my problem the equations are multiples of each other and not exact numbers. That's why the fitting is successful with the homogeneous representation that depends on the ratio. It fails otherwise.

After noticing that the correct solution norm is bad for the inhomogeneous equations i figured out the problem.

Answer 1

In using least squares, you normally want the residuals from the various equations to be indpendent of each other. If your $q_{i}$ vectors are imprecise measurements, then this will introduce correlation between the residuals in the pair of equations invovling $q_{i}$.

When you say that you get a "good fit" using the homoegneous system, how well does the solution satisfy the original nonhomogeneous system of equations?

Answer 2

I don't think this is very mysterious. It looks to me like your data $(q_i(1),q_i(2),q_i(3))$ fall nearly on some plane through the origin, corresponding to a homogeneous equation $a q_i(1) + b q_i(2) + c q_i(3) = 0$.
The fact that you write $a = 3 x_1 + x_2$, $b = 3 x_3 + x_4$, $c = 3 x_5 + x_6$ is irrelevant. But if you try fitting them to a plane not through the origin, such as $x_1 q_i(1) + x_3 q_i(2) + x_5 q_i(3) = -1$, you'll get a bad fit, because they are not close to such a plane.