You simply need to recognize that this is already a linear least squares problem and then work through the notation to put it into a form that you can give to a suitable solver.

Begin with the problem in the form

$
\min_{X} \sum_{i=1}^{m} \| a_{i} - X b_{i} \|_{2}^{2}
$

We'll assume that $X$ is a square matrix of size $n$ by $n$.

Next, look at an individual term
$
\| a - X b \|_{2}^{2}
$

Note that I've removed the subscript $i$ to simplify the notation in the following.

Reorganize $X$ by stacking columns of $X$ into a vector $x$, where $x_{1}=X_{1,1}$, $x_{2}=X_{2,1}$, $\ldots$, $x_{n}=X_{n,1}$, $x_{n+1}=X_{1,2}$, $\ldots$, $x_{n^{2}}=X_{n,n}$.

Now, write $Xb$ as $Ux$, where row $j$ of $U$ multiplied by $x$ gives the $j$th entry in $Xb$. The $j$th entry in $Xb$ comes from row $j$ of $X$ multiplied by the elements of $b$. That is, $\sum_{k=1}^{n} X_{j,k}b_{k}$ or $\sum_{k=1}^{n} x_{j+(k-1)n}b_{k}$. Thus

$
U_{j,j+(k-1)n}=b_{k}
$

for $k=1, 2, \ldots, n$ and $j=1, 2, \ldots, n$.

The remaining entriesin $U$ are 0's. Thus $U$ is a very sparse matrix.

Repeat this process for each term, $i=1, 2, \ldots, m$ and call the resulting $U$ matrices $U_{1}$, $U_{2}$, $\ldots$, $U_{m}$. You now have the problem in the form

$
\min \sum_{i=1}^{m} \| a_{i} - U_{i} x \|_{2}^{2}
$

Let

$
W=\left[
\begin{array}{c}
U_{1} \\
U_{2} \\
\vdots \\
U_{m}
\end{array}
\right]
$

Let

$v=\left[
\begin{array}{c}
a_{1} \\
a_{2} \\
\vdots \\
a_{m}
\end{array}
\right]
$

Now, your problem is a conventional least squares problem

$
\min_{x} \| v-Wx \|_{2}^{2}
$

You can express the solution to this least squares problem using the normal equations as:

$
x=\left( W^{T}W \right)^{-1} W^{T}v
$

However, in practice it is probably better to not explicitly form the $W^{T}W$ matrix and take its inverse. Given the sparsity of $W$, using an iterative method might be appropriate. There are also "sparse QR" factorization methods that might be appropriate.