A linear solver with optimal complexity $N^2$ will have to be applied $N$ times to find the entire inverse of the $N\times N$ real matrix $A$, solving $Ax=b$ for $N$ basis vectors $b$. This is a widely used technique, see for example Matrix Inversion Using Cholesky Decomposition, because it has modest storage requirements, in particular if $A$ is sparse. The Coppersmith–Winograd algorithm offers a smaller computational cost of order $N^{2.3}$, but this improvement over the $N^3$ cost by matrix inversion is only reached for values of $N$ that are prohibitively large with respect to storage requirements. An alternative to linear solvers with a $N^{2.8}$ computational cost, the Strassen algorithm, is an improvement for $N>1000$, which is also much larger than in typical applications.
So I would think the bottom line is, yes, linear solvers are computationally more expensive for matrix inversion than the best direct methods, but this is only felt for very large values of $N$, while for moderate $N\lesssim 1000$ the linear solvers are faster and have a much reduced storage requirement than direct solversmatrix inversion.