Geometric interpretations of matrix inverses

Question

$A$ is an invertible $n \times n$ matrix. Interpret each row of $A$ as a point in $\mathbb{R}^n$; then these $n$ points define a unique hyperplane in $\mathbb{R}^n$ that passes through each point (this hyperplane does not intersect the origin).

Under this geometric interpretation, $A^{-1}$ has an interesting property: the normal vector to the hyperplane is given by the row sums of $A^{-1}$ (i.e. $A^{-1} * 1$, where $1 = \langle 1, \dots, 1 \rangle^T$).

Within this same geometric interpretation of $A$, what other interesting properties does $A^{-1}$ have? Do the individual entries of $A^{-1}$ have geometric meaning? How about the column sums (besides the obvious row sums of $A^T$ intepretation)?

Answer 1

Here is one simple geometric relationship: if $X_1,\ldots,X_n$ are the rows of $A$, and $C_1,\ldots,C_n$ are the columns of $A^{-1}$, then the length of the column $C_i$ is equal to the reciprocal of the distance between $X_i$ and the hyperplane $V_i$ spanned by the other $n-1$ rows $X_1,\ldots,X_{i-1},X_{i+1},\ldots,X_n$ of $A$. This is because $C_i$ is orthogonal to $V_i$ and has an inner product of $1$ with $X_i$. If one then square sums in $i$, we obtain the negative second moment identity

$$ \sum_{i=1}^n \sigma_i(A)^{-2} = \sum_{j=1}^n \hbox{dist}(X_j,V_j)^{-2}$$

where $\sigma_1(A),\ldots,\sigma_n(A)$ are the singular values of $A$, which turns out to be a useful identity in random matrix theory (see e.g. this blog post of mine). In particular, it highlights the importance of understanding the distance between a row and the hyperplane spanned by the other rows if one is to get some control on the small singular values.