A doubly stochastic matrix is a square matrix with non-negative real entries where the sum of each row is $1$ and the sum of each column is $1$. A pairwise averaging matrix is a matrix of the form $tA+(1-t)B$ where $t\in[0,1]$ and $A,B$ are permutation matrices (recall that the permutation matrices are precisely the doubly stochastic matrices where each entry is either a $0$ or a $1$; clearly the $n\times n$-permutation matrices can be put into a one-to-one-correspondence with permutations of $\{1,\dots,n\}$)
Let $N_{n}$ be the $n\times n$-matrix where each entry is $1/n$. Does there exist a neighborhood $U$ of $N_{n}$ where if $A$ is a doubly stochastic matrix with $A\in U$, then $A$ can be written as a product of pairwise averaging matrices?
There are plenty of examples of doubly stochastic matrices that cannot be written as products of pairwise averaging matrices. See this similar question for more details.
$N_{n}$ itself can be written as a product of pairwise averaging matrices. Here is one way to get factorizations.
Let $DS_{n}$ denote the space of all $n\times n$-doubly stochastic matrices. Let $F_{n}$ be the set of all matrices that can be written as a product of pairwise averaging matrices. Then $F_{n}\subseteq\overline{F_{n}^{\circ}}$ where the closure and interior are taken in the space $DS_{n}$. Here is a proof. I have tried unsuccessfully to extend this proof to show that $N_{n}\in F_{n}^{\circ}$.