Why were matrix determinants once such a big deal? - MathOverflow

Why were matrix determinants once such a big deal?

2010-08-18T17:09:48Z

I have been told that the study of matrix determinants once comprised the bulk of linear algebra. Today, few textbooks spend more than a few pages to define it and use it to compute a matrix inverse. I am curious about why determinants were such a big deal. So here's my question:

a) What are examples of cool tricks you can use matrix determinants for? (Cramer's rule comes to mind, but I can't come up with much more.) What kind of amazing properties do matrix determinants have that made them such popular objects of study?

b) Why did the use of matrix determinants fall out of favor? Some back history would be very welcome.

Update: From the responses below, it seems appropriate to turn this question into a community wiki. I think it would be useful to generalize the original series of questions with:

c) What significance do matrix determinants have for other branches of mathematics? (For example, the geometric significance of the determinant as the signed volume of a parallelepiped.) What developments in mathematics have been inspired by/aided by the theory of matrix determinants?

d) For computational and theoretical applications that matrix determinants are no longer widely used for today, what have supplanted them?

Answer by J. M. for Why were matrix determinants once such a big deal?

2010-08-18T17:25:30Z

At least on the numerical front: the computation of determinants is prone to overflow for large enough matrices, which is why libraries like LINPACK made provisions for separately computing the mantissa and the exponent (see this for instance).

Cramer really isn't "cool" for solving linear equations either; as may have been mentioned before, for a general dense matrix, it takes O(n³) operations at best to compute a determinant (essentially through Gaussian elimination with some form of pivoting), and O(n!) at worst if you insist on cofactor expansion. Since you need to compute n+1 determinants to solve a linear system, it takes more effort to use Cramer than if you had used Gaussian elimination directly to solve your linear system.

Another application I have seen for determinants is as a check for the positive definiteness of a symmetric matrix through computing successive determinants of leading submatrices (this has applications in signal processing for instance); this too is slow compared to using e.g. Cholesky decomposition for checking if a matrix is positive definite.

On the matter of computing eigenvalues, determinants too were once used for the computation of the characteristic polynomial (for successive use with an appropriate rootfinding routine); nowadays with more stable modern algorithms like QR for eigenvalue computations, one no longer bothers with generating the characteristic polynomial.

Answer by Helge for Why were matrix determinants once such a big deal?

2010-08-18T18:10:06Z

Here is one cute analysis application. Consider a continuously differentiable function $A(x)$ on a real argument and taking values in the $N \times N$ matrices. Furthermore suppose that $\lambda(0)$ is a simple eigenvalue of $A(x)$, i.e. $dim(\ker( (A - \lambda(0))^N )) = 1$. Then there exists a small interval $I$ containing $0$ and a function $\lambda(x)$ such that $\lambda(x)$ such that $\lambda(x)$ is an eigenvalue of $A(x)$.

The proof is just applying the implicit function theorem to $$ f(x,E) = \det(A(x) - E), $$ since $\lambda(0)$ being simple implies that $\partial_E f(0,\lambda(0)) \neq 0$.

Furthermore, using the characterization of eigenvalues using the determinant, one can get some geometric information on how the curve $\lambda(x)$ looks.

Answer by Thierry Zell for Why were matrix determinants once such a big deal?

2010-08-18T19:03:30Z

Very good question, I think. There is indeed a backlash against determinants, as evidenced for instance by Sheldon Axler's book Linear Algebra Done Right. To quote Axler from this link,

The novel approach used throughout the book takes great care to motivate concepts and simplify proofs. For example, the book presents, without having defined determinants, a clean proof that every linear operator on a finite-dimensional complex vector space (or on an odd-dimensional real vector space) has an eigenvalue.

Indeed. If you think that determinants are taught in order to invert matrices and compute eigenvalues, it becomes clear very soon that Gaussian elimination outperforms determinants in all but the smallest instances.

On the other hand, my undergraduate education spent a tremendous amount of time on determinants (or maybe it just felt that way). We built the theory (from $\dim \Lambda^n(K^n)=1$), proved some interesting theorems (row/column expansion, bloc-determinants, derivative of det(A(x))), but never used determinants to perform prosaic tasks. Instead, we spent a lot of time computing $n \times n$ determinants such as Vandermonde (regular and lacunary), Cauchy, circulant, Sylveser (for resultants)... and of course, giving a few different proofs of the Cayley-Hamilton theorem!

What's the moral of the story? I think it's twofold:

Determinants are mostly a concern from a theoretical point of view. Computationally, the definition is awful. The most sensible thing to do to compute a determinant is to use Gaussian elimination, but if you're going to go through that bother, chances are that it's not really the determinant that you're after but rather something else that elimination will give you.
Determinants are a fertile ground to get to grips with a lot of really fundamental mathematical tools that a student of abstract mathematics should know backwards and forward. If you do everything I described above, you must learn deep results about the structure of the symmetric group $S_n$ (and more generally about multilinear forms), 10 flavors of induction, practical uses of group morphisms (from $GL_n(K)$ to $K^\star$). And of course, the existence of determinants itself is crucial to the more theoretical developments such a student will encounter later on.

I've had pure math undergrads who had learned linear algebra from Axler's book. They knew how to compute a determinant. They had no idea why anyone would want to. So determinants are still a big deal, but just for the right audience: I'm perfectly fine with most scientist and engineers ignoring determinants beyond $3\times 3$. Mathematics students, especially those with a theoretical bent, can learn a lot from determinants.

Answer by Deane Yang for Why were matrix determinants once such a big deal?

2010-08-18T19:15:23Z

Determinants are very important in almost every part of mathematics that I know. I wish I understood them better. But working directly with matrix determinants is very cumbersome, computation-intensive, and often unenlightening. So indeed they are not taught as much as they used to be. For pure mathematicians they have been replaced by exterior algebra, which is a much more elegant setting. And, as Spiro mentions, geometers work with signed volume forms, which turn into determinants when written with respect to a basis.

Even in computational mathematics, there is something called geometric algebra, which uses exterior, quaternionic and Clifford algebras to encode all the algorithms in an elegant and yet computationally efficient manner. I'm under the impression that video game programmers find this very useful.

Answer by Bruce Westbury for Why were matrix determinants once such a big deal?

2010-08-18T20:03:36Z

The Schur functions were defined using determinants. There is the classical definition as a determinant divided by the Vandermonde determinant. There is also the Jacobi-Trudi formula which expresses the Schur function as a determinant of complete homogeneous functions (or elementary functions). This came well before any theory of linear algebra.

Answer by KConrad for Why were matrix determinants once such a big deal?

2010-08-18T20:41:58Z

To confirm your remark that determinants were once the bulk of linear algebra, you should find in a university library the books by Thomas Muir on determinants. They're enormous! (You can also find them on Google Books, but physically holding them leaves a greater impression, especially if they are in your lap.) It reminds me of a comment of Serre: "Forgetting is a very healthy activity."

Answer by Andrew for Why were matrix determinants once such a big deal?

2010-08-18T21:36:38Z

For the reasons of reputation points, I am unable to comment on KConrad's post. I up-voted his response, however, because he does make a great point about the Muir book(s). This one in particular is of great interest if you have any thoughts as to wondering why determinants (and development of them) was/is important:

The Theory of Determinants in the Historical Order of Development

I've gone through parts and it's dry, but thorough.

Answer by Pace Nielsen for Why were matrix determinants once such a big deal?

2010-08-18T22:33:42Z

To answer (c):

I can't count how many times I've used the fact that for Vandermonde matrices, the determinant is what it is.

Answer by Richard Stanley for Why were matrix determinants once such a big deal?

2010-08-19T01:46:04Z

Dedekind raised the question of computing the determinant of the multiplication table of a finite group (regarding the group elements as commuting indeterminates). The abelian case was well-understood. Frobenius was a master of determinants and created group representation theory in order to answer Dedekind's question. Frobenius' approach to group representations was based on determinants. This is explained by T. Hawkins in Arch. History Exact Sci. 27 (1970/71), 142-170; 8 (1971/72), 243-287; 12 (1974), 217-243.

I should also point out that the evaluation of determinants is alive and well within combinatorics. Often a number or generating function can be expressed as a determinant. This is considered a "nice" answer because determinants can sometimes be evaluated explicitly, and in any event can be evaluated quickly and have many other useful properties. See for instance the work of Krattenthaler, especially http://www.mat.univie.ac.at/~kratt/artikel/detsurv.html and the sequel http://www.mat.univie.ac.at/~kratt/artikel/detcomp.html.

Answer by Alexander Woo for Why were matrix determinants once such a big deal?

2010-08-19T02:57:47Z

Warning: I am basically just speculating, and not commenting with actual knowledge of the history.

I suspect that a lot of the nineteenth century work on determinants was motivated by invariant theory. Before Hilbert proved abstractly the finite generation of invariants, there was a small industry trying to explicitly compute invariants (for example the projective invariants of the action of $GL_2$ acting on binary forms) with the aim of giving a constructive proof that they were finitely generated.

These invariants often have a basis consisting of various sorts of determinantal expressions, and if you want to prove finite generation, you have to construct ways of taking certain determinantal expressions and writing them in terms of other determinantal expressions.

Within a decade or so of Hilbert's paper, people generally lost interest in constructive invariant theory. (After all, the abstract methods answered the most interesting questions and seemed much more likely than constructive methods to answer the most interesting remaining questions.)

I have wondered whether all the work on syzygies of determinantal varieties actually reduces to identities which were well known (to the right people) in the 19th century.

Answer by Matus Telgarsky for Why were matrix determinants once such a big deal?

2010-08-19T03:00:39Z

I think the multivariate change-of-variable formula in integration (i.e. involving the determinant of the Jacobian) is still rather indispensable. The treatment I'm most familiar with is in Folland, where, as far as I recall, it is only used to construct integration in polar coordinates (and I think there was only one exercise, concerning a further extension).

One could perhaps say that the trick of computing the normalization to a Gaussian random variable, by way of passing through polar coordinates, uses determinants. EDIT: this fact also provides an immediate explanation for the presence of the determinant in the denominator of a multivariate Gaussian (and by positive semi-definiteness of the covariance, that the square root makes sense).

Answer by Denis Serre for Why were matrix determinants once such a big deal?

2010-09-22T10:29:04Z

I don't think that determinants is an old fashion topic. But the attitude towards them has changed along decades. Fifty years ago, one insisted on their practical calculation, by bare hands of course. This way of teaching linear algebra has essentially disappeared. But the theoretical importance of deteminants is still very high, and they are usefull in almost every branch of Mathematics, and even in other sciences. Let me give a few instances where determinants are unavoidable.

Change of variable in an integral. Isn't the Jacobian of a transformation a determinant?
The Wronskian of solutions of a linear ODE is a determinant. It plays a central role in spectral theory (Hill's equation with periodic coefficients), and therefore in stability analysis of travelling waves in PDEs.
A well-known proof of the simplicity of the Perron's eigenvalue of an irreducible non-negative matrix is a very nice use of the multilinearity of the determinant.
The $n$th root of the determinant is a concave function over the $n\times n$ Hermitian positive definite matrices. This is at the basis of many development in modern analysis, via the Brunn-Minkowski inequality.
In combinatorics, determinants and Pfaffians occur in formulas counting configurations of lines between sets of points in network. D. Knuth advocates that there are no determinants, but only Pfaffians.
Of course, the eigenvalues of a matrix are the roots of a determinant, the characteristic polynomial. In control theory, the Routh-Hurwitz algorithm, which checks whether a system is stable or not, is based on the calculation of determinants.
As mentioned by J.M., Slater determinants are used in quantum chemistry.
Frobenius Theory provides an algorithm for classifying matrices $M\in M_n(k)$ up to conjugation. It consists in calculating all the minors of the matrix $XI_n-M\in M_n(k[X])$ (these are determinants, aren't they?), then the g.c.d. of the minors of size $k=1,\ldots,n$ for each $k$. This is the theory of similarity invariants, which are polynomials $p_1,\ldots,p_n$, with $p_j|p_{j+1}$ and $p_1\cdots p_n=P_M$, the characteristic polynomial. If one goes further by decomposing the $p_j$'s (but this is beyond any algorithm), one obtains the theory of elementary divisors.
If $L$ is an algebraic extension of a field $K$, the norm of $a\in L$ is nothing but the determinant of the $K$-linear map $x\mapsto ax$. It is an element of $K$.
Kronecker's principle characterizes the power series that are rational functions, in terms of determinants of Hankel matrices. This has several important applications. One is the proof by Dwork of Weil's conjecture that Zeta functions of algebraic curves are rational functions. Another one is Salem's theorem: if $\theta>1$ and $\lambda>0$ are real numbers, such that the distances of $\lambda\theta^n$ to ${\mathbb N}$ are square summable, then $\theta$ is an algebraic number of class $S$.
Above all, invertible matrices are characterized by their determinant: it is an invertible scalar. This is true when the scalars belong to a unit commutative ring. Besides, the determinant is the unique morphism ${\bf GL}_n(A)\mapsto A^*$ ; it therefore plays the same role in the linear group as that played by the signature in the symmetric group $\frak S_n$.
Powers of the determinant of $2\times2$ matrices appear in the definition of automorphic forms over the Poincaré half-plane.
See also the answers to JBL's question, Wonderful applications of the Vandermonde determinant, http://mathoverflow.net/questions/43538

Of course, this list is not exhaustive (otherwise, it should be infinite). I do teach Matrix Theory, at Graduate level, and spend a while on determinant, even if I rarely compute an exact value.

Answer by Vamsi for Why were matrix determinants once such a big deal?

2010-12-22T06:57:54Z

1) The Chern-Weil theory of characteristic classes is built upon determinants of functions of curvature forms of vector bundles. 2) Feynman path integrals require determinants (but typically in infinite dimensions).

Answer by BSteinhurst for Why were matrix determinants once such a big deal?

2010-12-22T21:33:10Z

From the realm of probability there are determinental and permanental processes. Terry Tao has a nice post about determinental processes here. For instance

"Examples of processes known to be determinantal include non-intersecting random walks, spectra of random matrix ensembles such as GUE, and zeroes of polynomials with gaussian coefficients."

I've not worked with these processes myself but I've heard enough seminar talks using them to say there is plenty of interest out there.

Answer by Jose Brox for Why were matrix determinants once such a big deal?

2010-12-23T16:46:05Z

The simple fact that the invertible square matrices of order $n$ are precisely those which have nonzero determinant has a real bunch of theoretical applications (in ring theory, topology, differential geometry, etc).

For example, if you consider the general linear group $GL(n, \mathbb{R})$ as a Lie group, to see that its associated Lie algebra is (isomorphic to) $gl(n, \mathbb{R})$, an essential step is to get the equivalence of tangent planes $T_e(GL(n,\mathbb{R})) \cong T_e(gl(n,\mathbb{R}))$ (where $e$ is the neutral element). But this is trivial if we have in mind that $GL(n,\mathbb{R}) = \{A\in gl(n,\mathbb{R}) : det(A) \neq 0 \}$ and that $det$ is continuous, because then $GL(n,\mathbb{R})$ is automatically an open set of $gl(n,\mathbb{R})$.

Answer by Kjetil B Halvorsen for Why were matrix determinants once such a big deal?

2012-06-07T02:13:32Z

Just an example from applied statistics where determinants are unavoidable (because they are used to define the relevant concepts. In design of experiments, one wants to choose "optimal" values of the regressor variables $x_1, \dots x_p$ used to estimate a linear regression model $Y=X \beta + \epsilon$, where $X$ is the design matrix, an $n \times (p+1)$ matrix with the first column consisting of ones, and row $i$ being the covariates used for experimental run $i=1, \dots , n$.

D-optimality defines the optimal design to be the one maximizing the determinant $\det X^T X$, under the practical restriction that the regressor values x's must belong to some set where the experiment is feasible to do.

One way of justifying this is that the requirement is equivalent to minimizing the volume of confidence ellipsoids, calculated under the normal model. Mostly, numerical optimization is used to construct such designs. (although there are many theoretical papers solving toy models).

Answer by Sigurd Angenent for Why were matrix determinants once such a big deal?

2012-09-25T16:38:46Z

An elementary use that was in the books on determinants (not linear algebra) I read as a student long ago: you can use them to write cute equations for objects in elementary (Euclidean or projective) geometry. For instance, the equation for the circle through three points in the plane is $$ \left| \begin{matrix} x^2+y^2 & x & y & 1 \cr a_1^2+a_2^2 & a_1 & a_2 & 1\cr b_1^2+b_2^2 & b_1 & b_2 & 1\cr c_1^2+c_2^2 & c_1 & c_2 & 1 \end{matrix} \right| = 0 $$ If the coefficient of $x^2+y^2$ is non zero then it is obviously the equation for a circle, and the three points $A, B, C$ obviously lie on the circle.
The coefficient of $x^2+y^2$ is $$ \left| \begin{matrix} a_1 & a_2 & 1\cr b_1 & b_2 & 1\cr c_1 & c_2 & 1 \end{matrix} \right| $$ and has to be nonzero. Since the equation for the line through $(b_1, b_2)$ and $(c_1, c_2)$ is $$ \left| \begin{matrix} x_1 & x_2 & 1\cr b_1 & b_2 & 1\cr c_1 & c_2 & 1 \end{matrix} \right|=0 $$ the coefficient of $x^2+y^2$ is nonzero if the three points do not lie on a straight line.

In the same vein, the linear homogeneous differential equation satisfied by $y_1(x)$ and $y_2(x)$ is $$ \left| \begin{matrix} y''(x) & y'(x) & y(x) \cr y_1''(x) & y_1'(x) & y_1(x) \cr y_2''(x) & y_2'(x) & y_2(x) \end{matrix} \right| = 0 $$

Answer by Jorge Stolfi for Why were matrix determinants once such a big deal?

2012-09-25T17:20:20Z

The area of an arbitrary triangle is most easily computed from its vertex coordinates as a 3x3 determinant (no square roots, trig, etc.). This generalizes to 3D (volume of a tetrahedron) and higher dimensions.

The cross product of two vectors a,b (essential tool in mechanics) is easily understood and memorized once one observes that its coordinates are 2x2 minor determinants of a 2x3 array formed by a and b. This too generalizes and provides a definition for cross product of n-1 vectors in n dimensions.

More generally, determinants combined with homogeneous coordinates are all one needs to derive elegant formulas for the basic operations of n-dimensional projective geometry, such as plane through 3 points, intersection of 3 planes. They also provide a coordinate representation (Pluecker coordinates) for lines in 3-space, or more generally for k-dim subspaces of n-dim prijective space.

The sign of the determinant of a matrix tells whether the rows are a left- or right-handed frame, and whether the corresponding linear map preserves or reverses orientations.

Determinants are more efficient than Gaussian elimination to compute the inverse of a 2x2 or 3x3 matrix, perhaps even 4x4. Unlike standard G.E. these formulas do not require division and may require fewer bits per number in intermediate results.

Answer by Jim Dukelow for Why were matrix determinants once such a big deal?

2012-09-25T17:46:03Z

I quickly scanned the answers and didn't find the following:

Square matrices of order n can be considered to be embedded in R^(nxn). The determinant is a continuous function of the entries of the matrix, so the singular matrices are det^(-1)(0) and are therefore a closed set in R^(nxn). Thus, there will be non-singular matrices arbitrarily close to a singular matrix in any convenient metric on R^(nxn). Of course, they will have pretty ugly condition numbers.