I am reading Landsberg's "Tensors: Geometry and Applications". Here he mentions tensor formulation of Strassen's algorithm and shows that the rank of Strassen's matrix multiplication tensor is $7$ and $7$ is the lower bound for any such tensor and hence one needs $7$ multiplications for $2 \times 2$ matrix multiplication. My question is the following: is $7$ the absolute lower bound among all algorithms or just those that can be formulated via tensors?

This $7$ is an absolute lower bound. The result is due to Hopcroft and Kerr "On minimizing the number of multiplications necessary for matrix multiplication." SIAM J. Appl. Math. (1971) and Winograd "On multiplication of $2\times 2$ matrices." Linear Algebra and Appl. (1971). [The former assume that entries of the matrices might not commute; while the latter gets the bound even assuming commutativity of the entries.] A lot more recently Landsberg showed that not only the rank but even the border rank of multiplication of $2 \times 2$ matrices is $7$, meaning very roughly that also small perturbations cannot lead to a smaller rank and thus saving of a multiplciation (for "approximate" calculations), assuming bilinearity of the algorithm. The paper establishing this is Landsberg "The border rank of the multiplication of $2\times 2$ matrices is seven", Journal Amer. Math Soc. 2006. The introduction also discusses your question. See the link at the end for the respective volume of the journal, I think the article is free: http://www.ams.org/journals/jams/20061902/home.html 


Quid's reply already gives you a complete answer for the $2\times 2$ case; however, there is an additional consideration that adds to the picture of the optimal exponent of matrix multiplication. One can prove that, for any $t$, if there exists a straightline program that multiplies $n\times n$ matrices in $O(n^t)$ ops (without any bilinearity assumption or restriction on the operations made: divisions, additions, products of three more terms, everything is allowed), then there is a bilinear algorithm (i.e., one that can be "formulated with tensors") that achieves the same asymptotic complexity $O(n^t)$. So one does not lose generality by restricting to bilinear algorithms, when looking for the optimal exponent of matrix multiplication. I had this same doubt myself a few years ago, and I found a proof of this statement in BorodinMunro, The computational complexity of algebraic and numeric problems. 

