This is my first MathOverflow post: I apologize if my message is lacking of something. I also posted this question in Math StackExchange, but as I haven't seen an answer I post it here.

Suppose you have system of equations: $$Ax = b \rightarrow [A_{1}, A_{2},...,A_{M}]^{T}x = [b_{1},...,b_{M}]^{T} \: \: \: (1)$$

where each and all matrices $A_{1},...,A_{M}$ $\in \mathbb{R}^{S \times L}$. We call this system of equations the whole system.

In the same manner, each and all $b_{1},..., b_{M} \in \mathbb{R}^{S \times 1}$ and $x \in \mathbb{R}^{L \times 1}$.

As you see in (1), $A$ is composed by matrices $A_{1},...,A_{M}$ as well as $b$ by vectors $b_{1},...,b_{M}$. For each of these $A_{i}$ and $b_{i}$ you can write down equations:

$$A_{i}x_{i} = b_{i}, \: \; i = 1,...,M \: \: \: (2)$$

We call each of these equations a partial system.

By Least Squares you know the solutions $x_{1},...,x_{M}$ to each of the partial systems in (2). However, by restrictions you CAN'T solve directly the whole system of equations in (1) through Least Squares.

Let's suppose we know the error of each of the partial solutions $x_{i}$ with respect to the whole solution $x$. That is to say that we know:

$$ e(x_{i}) = || x_{i} - x||_{p} \: \: \: (3)$$

where in (3) $||\cdot||_{p}$ is some norm to be defined.

We want to build $x$ from $x_{1},..,x_{M}$ in such a way that the more $x_{i}$ that we include into the solution we build, the smaller the error with respect to $x$, and all the $x_{i}$ have the same importance.

In other words, we want to produce a sequence of functions $y_{1},...y_{M}$ as follows:

$$y_{1} = f(x_{k_{1}}), \: \: \: f: \mathbb{R}^{L} \rightarrow \mathbb{R}^{L} $$ $$y_{2} = f(x_{k_{1}},x_{k_{2}}), \: \: \: f: \mathbb{R}^{L} \times \mathbb{R}^{L} \rightarrow \mathbb{R}^{L}$$ $$y_{3} = f(x_{k_{1}},x_{k_{2}},x_{k_{3}}), \: \: \: f: \mathbb{R}^{L} \times \mathbb{R}^{L} \times \mathbb{R}^{L} \rightarrow \mathbb{R}^{L}$$ $$.$$ $$.$$ $$.$$ $$y_{M} = f(x_{k_{1}},...,x_{k_{M}}) = x, \: \: \: f: \mathbb{R}^{L} \times ... \times \mathbb{R}^{L} \rightarrow \mathbb{R}^{L}$$

such that $e(y_{M}) < e(y_{M-1}) < ... < e(y_{2}) < e(y_{1})$.

The indexes $k_{1},...,k_{M}$ represent which of the solutions $x_{i}$ are taking into account to build each $y$. We use those indexes to say that any of the partial solutions $x_{i}$ is NOT more important than any other.