Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

The cost of solving a linear system ("exactly") with Gauss Elimination and other similar methods with a few right hand side and where the matrix has no structure is $\mathcal{O}(N^3)$ where $N$ is the system size.

I am wondering about the lower bound for solving a linear system. An obvious lower bound is $\mathcal{\Omega}(N^2)$ (since the information content is $\mathcal{O}(N^2)$). Are there better lower bounds other than $\mathcal{\Omega}(N^2)$ for solving the linear system? Is there a way to prove that the lower bound of $\mathcal{\Omega}(N^2)$ can never be hit for a matrix with no special structure? (assume that we are solving a system with only one/few right hand side).

Also are there other algorithm which solve these system "exactly" whose cost is less than $\mathcal{O}(N^3)$? I am aware of Strassen algorithm which perform matrix multiplications in $\mathcal{O}(N^{\log_27})$. I assume this can be used to solve a linear system in $\mathcal{O}(N^{\log_27})$. (?)

(The system has no special structure. I am not worried about the stability and other numerical intricacies of the method as of now. I would appreciate if someone could point to some work done in this regard.)

Thanks

share|improve this question

2 Answers 2

up vote 2 down vote accepted

Any $O(N^t)$ algorithm for matrix multiplication yields a corresponding $O(N^t)$ algorithm for matrix inversion. There are not any non-trivial lower bounds for matrix multiplication. It is believed that there exist $O(N^{2 + \epsilon})$ algorithms for any $\epsilon > 0$, but as far as I know there are not believed to be $O(N^2)$ algorithms.

share|improve this answer
1  
The wikipedia article is very good: en.wikipedia.org/wiki/… –  Igor Rivin Jan 19 '11 at 23:28
    
Thanks David and Igor. Yes I am aware that solving a system and multiplication of two matrices have the same order of cost. lehigh.edu/~gi02/m242/08linstras.pdf. Anyway I will look if someone else answers this question or provides other useful insights before accepting the answer. –  user11000 Jan 20 '11 at 1:58
    
Also, you should remember that this is for solving/multiplying under the assumption that arithmetic operation is unit cost. This is not true if your are trying to solve equations over $\mathbb{Q}$ or $\mathbb{Z}.$ –  Igor Rivin Jan 20 '11 at 4:57

Let me take this opportunity to make a comment that talks about the upper bound, not the lower.

Actually, let me remind you that it is a common false belief that Gaussian Elimination has $O(n^3)$ complexity. See the nice question with its answers here at cstheory

This misbelief happens because even though GE requires $O(n^3)$ arithmetic operations, if not done properly, there can be massive intermediate coefficient growth, which renders judging the true complexity of GE a difficult task.

You might also enjoy looking here at the near linear time algorithms for solving special linear systems (example for diagonally dominant matrices). For general, non-structured matrices, the situation is less clear.

share|improve this answer
1  
Gaussian Elimination over a black-box field $F$ requires $O(n^3)$ arithmetic operations, where an arithmetic operation is defined to be an access to the arithmetic oracle for $F$. In some fields, such as finite fields of fixed size, there is no coefficient growth so this estimate is exactly right. For fields such as $\mathbf R$ or $\mathbf C$, typically one approximates these with floating-point representations. In this case, Gauss Elimination uses $O(n^3)$ flops (but may commit some inaccuracy in the result). –  David Harris Jan 20 '11 at 14:30
    
Notice that the OP specifically asks for "exact" solution. –  Igor Rivin Jan 21 '11 at 15:34
    
Good point Igor; somehow I oversaw that in my haste. But I guess then one could argue that if even an inexact solution cannot be done better than in some time, say T, then getting an exact solution should be ever harder. –  Suvrit Jan 21 '11 at 17:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.