The sum of the elements of a matrix $M$ is $e^T M e$, where $e$ is the vector of all ones.
So, instead of computing the inverse, you should solve the system $Ax=e$ and then compute $e^Tx$. This might look like a simple trick, but solving linear systems is faster than computing inverses in basically all settings.
Of course you should then use a method to solve this linear system which is appropriate to the matrix that you are dealing with (but there is a large amount of literature on that).
I don't think that you can get the quantity you want any faster than this, unless your matrix has very special properties.