I have a set $\{A_1, A_2, .. A_k\}$ of $n$ by $n$ real matrices and I know that they are 'perturbated versions' of a set of commuting matrices : $\{P_1,..,P_k\}$, by perturbated versions I mean that I have a bound on $\sum_i\A_iP_i\\leq \epsilon$. My problem is that I want to find an epsilon perturbation of $\{A_1,...,A_k\}$ so that they commute. It is clear that at least one such perturbation exists since they are perturbed versions of a commuting set. But my question is if there is any algorithm to find a permutation that makes the matrices in $A$ commute.

I think it really comes down to looking at the lattice of invariant subspaces for each $A_i$. I believe this is what Igor basically is saying. For a single real linear transformation $V \rightarrow V$ with distinct characteristic roots, $V$ decomposes as a sum of 1 and 2dimensional invariant subspaces, and these generate the lattice of invariant subspaces. When you perturb the matrix by a known amount, the subspaces can wander around over a certain range  how much they shift depends on how close together are the characteristic roots, and also on the angles between the subspaces. For simplicity, think of the case with all real distinct eigenvalues. The projectivized action, on $\mathbb{RP}^{n1}$, then has $n$ fixed points. The first derivative of the projectivized map is simple to determine, given the eigenvectors and eigenvalues  the eigenvalues of the derivative at a point for eigenvalue $\lambda$ are the ratios of the other eigenvalues to $\lambda$. Therefore, if you change the map by adding multiples of the other eigenvectors as a linear function of the projection to the eigenspace under consideration you shift your given eigenvector by a known linear function of the coefficients. If you have several matrices with all real distinct eigenvalues that are near enough to a commuting set of matrices, then presumably the eigenvectors are also near to each other, so you can tell which eigenvectors need to line up with each other. You have a good measurement of how hard it is to move each eigenvector in a given direction, so you can find a point that's approximately equally hard for each (do this separately, for each matching collection of eigenvectors.) Then adjust each of the eigenvectors to go to that point, and repeat on the other eigenspaces. When there are complex roots, there are 2dimensional invariant subspaces, which appear as invariant circles in $\RP^{n1}$. You can do much the same thing with these. When there are multiple real or complex roots, then it's necessary to look at largerdimensional invariant subspaces. One strategy is to first fix up invariant subspaces associated with the product of all factors of the characteristic polynomial whose roots are in a small neighborhood, or a pair of complex conjugate neighborhoods. The goal is to get such subspaces to agree with, to contain, or to be contained in similar suspaces for the other matrices. If there are repeated roots, then first see if there's a small perturbation that makes multiple eigenvectors (or multiple 2dimensional invariant subspaces)  when this is possible, it gives a bigger target for matching commuting matrices. If not, look at the largest invariant subspace it contains. I doubt if there's an easy way other than looking at these invariant subspace structures. Perhaps in the actual applications you have in mind, they're not so complicated. 


Well, the following is a nottoofast algorithm. Reduce all matrices to Jordan canonical form, since the matrices almost commute, their right eigenvectors are almost the same. Adjust the matrices of eigenvectors to actually BE the same (and in case of nontrivial jordan blacks perturb them so the matrix is actually diagonal. Jordan blocks and sets of nearby eigenvalues makes implementing this a nuisance, since the eigenvectors of a slightly perturbed diagonal matrix are very unstable  you will need to look at which subspaces those blocks span. 


First, let us consider a simpler setting. Let $A_1$ and $A_2$ be given as inputs. The aim is to find a matrix $B$ such that $A_1+B$ commutes with $A_2+B$, and that $B$ has small norm. The keyconstraint is thus: $(A_1+B)(A_2+B) = (A_2+B)(A_1+B)$, which simplifies to the worstcase (since no unique solution is possible) Sylvester equation $$(A_1A_2)B + B(A_2A_1) = A_2A_1A_1A_2$$ Thus, a formulation as an optimization problem could be $$ \min\quad \B\^2\qquad \text{s.t.}\quad (A_1A_2)B + B(A_2A_1) = A_2A_1A_1A_2. $$ In principle (if we use the Frobenius norm for $B$), this is just a leastnorm problem (similar to leastsquares). In general, if we have $A_1,\ldots,A_k$, then we will have a hugenumber of constraints (for all pairs of commutativity requirements), but the overall problem should still be essentially a leastnorm problem, and can be thus solved ``easily.'' 


I think there is enough here to qualify as an answer. Plus it is too long for a comment.
$$P_1=\left[ \begin {array}{cc} 20000&10000\\10000&10000 \end {array} \right]$$ and $P_2$ its square (actually, $1/10000$ times its square). Then I added a random integer in the range $100..100$ to each entry preserving symmetry (I told you it was a baby problem). Then I got numerical eigenvalues and eigenvectors for each matrix, averaged the eigenvectors to get the compromise eigenbasis, and transformed back. The six random pertubation integers came out to be $11,83,70$ for the first matrix and $15,82,8$ for the second. The result of this process was a pair of commuting matrices which differed from $A_1$ and $A_2$ by approximately $5.35,2.67,5.35$ and $16.23,8.1,16.23$. Then I asked MAPLE to do a MINIMZE problem in $\mathbb{R}^6$ finding the point on the surface $(r_1r_3)r_5=r_2(r_4r_6)$ closest to a given one (from the $A_i$). It failed to find an initial point but when I used my previous answer as an initial point it returned commuting matrices which differed from $A_1$ and $A_2$ by approximately $9.65,4.81,9.65$ and $3.18,1.59,3.18$ Disclaimer: I realize that the constraints would be a nightmare for a pair (let alone $k$) large matrices. I don't have any idea of the state of the art for numerical solvers. 

