Let there be a possibly overdetermined system AX = B, where B are some measured data, with a low noise level. To cancel out the measurement noise, and to allow for more unknowns, a large B is acquired, by making a lot of measurements, every time changing measurement conditions, represented by A, only by a very small step. Thus, subsequent rows in A are almost identical. But, rows that are far apart, are substantially different.
Thus, within such a system exist equations that are very similar to each other, so their solution hyperplanes are very close to each other.
Add a minimal noise, or even arithmetic round--off errors, and intersections of such similar hyperplanes can be practically random. Even worse, possibly with very large absolute values.
But various least squares solvers I tried take these random intersections into account, and return unusable values of X.
But, if the equations in the system are divided into a number of subsets of mutually dissimilar equations within each set, each subset solved individually by one of the same solvers, and then the solutions averaged into a single one, it turns out the averaged result can be usable.
The question is, are there solvers that already take into account the extreme similarity of certain pairs of equations within a system + some small noise.
Imagine a two--unknowns system with four equations, the four lines represented by the equations produce something like the # sign. The solver would average the four ~90 deg crossings in the center into the solution, but it would completely ignore the two near-infinity crossings of parallel lines.
Note that the measurements in B do not contain any outliers, the noise is very low, it's just the almost-linearly-dependent equations.
