Ways to Synthesize Topics in Linear Algebra

Question

Hello, I am currently studying linear algebra right now. In general, the material is pretty straight-forward but it doesn't seem particularly interesting. I suppose that the main thing that I am lacking is some sort of guiding principle from which to view the material.

As an example, in studying quantum mechanics, it is an fun exercise to look at particular extremes of a problem in order to gain some understanding of the situation. It requires some creativity to see what quantities can be manipulated for interesting results.

With this in mind, I wonder what you all think would be a good way to look at linear algebra.

I am looking for more general ideas, but in particular, I'm concerned with looking at Dual Spaces and linear functionals in a better motivated way.

Best, Alex

Answer 1

One of the tough things about linear algebra is that it is just so damn useful for so many people that courses end up having a lot of compromises. And because of its pervasiveness, there are several good perspectives from which one should view the material.

For example, it's one of the first math courses to really teach algorithmic thinking. It's amazing to me just how much of elementary linear algebra you can prove just with Gaussian elimination. Various modifications of the classical algorithm are used to solve some heavy duty real world problems. Another example (although not so pervasive as Gaussian elimination) is the Gram-Schmidt process.

But I understand completely where you are coming from. Linear algebra is not just a set of tools to compute certain things -- it's a whole new way of thinking. To me, linear algebra makes a lot of sense when you think of it as capturing some basic ideas of geometry. Given an inner product, you have some very natural notions -- spheres, lines, hyperplanes, ellipsoids, etc. Linear transformations can be naturally thought of as geometric operations -- shears, rotations, inversions, etc. Some of the structure theorems for linear transformations have a geometric interpretation. One interpretation of the spectral theorem is if you have a symmetric operator A on a finite dimensional real vector space then this gives you a quadratic form f(t) = <t,At>. This function, restricted to the unit sphere, takes on a maximum value f(p) for some p, and it turns out that p is an eigenvector of A with eigenvalue f(p) (which is in fact the largest eigenvalue). You can get the next eigenvalue/eigenvector pair by considering the same maximization problem restricted to the orthogonal complement of f(p), and then keep repeating this, etc. This really blew my mind the first time I saw it, I'm pretty sure it's in Lang's Linear Algebra book. Another example of geometry coming in is the SVD... given any linear transformation, the image of a unit sphere under this transformation is an ellipsoid, and the lengths of the principal semi-axes are the singular values. Check wikipedia for the rest.

As for your remark about linear functionals, I'll chime in and say that one can naturally think of them as somehow measuring something. For example, the Riemann integral over some fixed set is a linear functional (on the vector space of riemann integrable functions). For a slightly different example, the Riesz representation theorem says that in any Hilbert space (complete inner product space. In finite dimensions, real or complex vector space have such a structure) a linear functional is of the form f(x) = <x,y> for some vector y, and intuitively you can imagine this as measuring (up to some scalar) the magnitude of vectors in the direction of y.

Answer 2

I list here some fun exercises that can be done. I made it community wiki so you can add yours to it too.

• Maybe it is too obvious, but do everything basis-free.
• Investigate if each result holds for real, complex, and other base fields. It is also worth thinking about what happens when the "base field" is just a ring (then the linear space becomes a module).
• Try to generalize the result for infinite dimensional cases, and identify what assumptions you need in order for the result to still hold.

Answer 3

Personally, I found myself in the same situation as you when I was studying linear algebra, and I never ended up understanding it very thoroughly at first because I could not find motivation to study all these seemingly unrelated theorems.

However, about a year after I had taken a course on linear algebra I realized how useful it can be, seeing it pop up in different fields like Google's PageRank algorithm, Markov Chains, approximating PDE's numerically, and SVD decomposition amongst a great deal of other things. I took it upon myself to re-educate myself more thoroughly in the subject and to do that I decided to find a good book on the subject. I found the book by Sheldon Axler called Linear Algebra Done Right and I can say that this book truly "does it right."

That is almost all of the introductory material is crystal clear and the first half of the book is basically working up to the first major result, the spectral theorem which is the NxN version of the incredibly useful SVD decomposition in practice. In fact, most of the book does not even consider matrices but is far more concerned with linear transformations which I feel is far more pertinent to the discussion of linear algebra than considering the properties of matrices.

Answer 4

When you first start learning about vector spaces, you tend to think of vectors as just being some ordered collection of numbers. This is very natural, since the first vector spaces you deal with are $\mathbb R^n$ and $\mathbb C^n$.

Then, at some point, someone suggests to you that your vectors don't have to be collections of numbers -- they could be continuous functions! It makes sense to add two functions in $n$ real variables or to scale such a function by a real number, and each function has an additive inverse, and so forth, so they form a real vector space. All the real (finite-dimensional) vector spaces you had encountered before were isomorphic to $\mathbb R^n$ for some $n$. Now, have we discovered a "genuinely new" vector space, or is this just $\mathbb R^n$ in another guise?

Well, things are a little wild if we let our functions be anything at all; let's look at the vector space formed just by the linear maps from $\mathbb R^n$ to $\mathbb R$. Now you can fix a basis for $\mathbb R^n$ and look at the dual basis, and convince yourself that the dual basis really is a basis of the vector space of linear maps. Finally, you just have to convince yourself that the map taking a basis vector $b_i$ of $\mathbb R^n$ to its dual $b_i^*$ is actually an isomorphism. So any time we want an $n$-dimensional real vector space, we can use $(\mathbb R^n)^*$ instead of $\mathbb R^n$, which is often convenient.

Chances are, the dual space wasn't motivated this way to you -- people often start with a basic vector space like $\mathbb R^n$ and dualize it for no apparent reason, except to show you that you get a vector space. I think it's more informative to start with the space of linear maps, since that's a very natural space to discover.

(Of course, you may have learned already that every finite-dimensional real vector space is isomorphic to $\mathbb R^n$ for some $n$. But this approach really lets you get your hands on the spaces.)

Answer 5

Linear algebra is relevant to pretty much every branch of mathematics. I think it'll make more sense once you see examples of it being relevant to every branch of mathematics.

Here's an example that shows why duals and linear functionals are important in geometry. Suppose you investigate a curve $f(x, y) = 0$ in $\mathbb{R}^2$ (say) and at $(0, 0)$ it has the form $ax + by + \text{higher order terms}$. When we perform this operation of "ignoring higher order terms" what we get is a vector space spanned by $x$ and $y$ called the cotangent space. The dual of this space is the tangent space. Why? We should think of tangents as defining "infinitesimal vectors" pointing from a point, and then elements of the form $cx + dy$ in the cotangent space define linear functionals which act on those tangents. The distinguished element $ax + by$ of the cotangent space is then orthogonal to the tangent line.

---

Okay, that was too specific. Let me talk from first principles. The first thing you always want to do when you have a bunch of objects is to specify what the morphisms between them are so you have a category. For vector spaces the morphisms are linear transformations - that's obvious enough. Now, it turns out that even if you aren't willing to use a basis, you can still describe linear transformations with "coordinates" in a certain sense. The way you do that is with the tensor product and the dual space: it turns out that $\text{Hom}(A, B)$ (the vector space of all linear transformations from $A$ to $B$) is naturally isomorphic to $A^{\ast} \otimes B$ (in the sense that one doesn't have to pick a basis to exhibit the isomorphism). So if you want to understand linear transformations, it suffices to understand the dual space and the tensor product.

The relationship is maybe easier to see if you pick bases $e_i, f_j$ of $A, B$ first: usually one presents a linear transformation $T$ as a bunch of coefficients $T_{ij}$ describing what the coefficient of $f_j$ in $T(e_i)$ is. Now, the basis $e_i$ determines what's called a dual basis $e_i^{\ast}$ on $A^{\ast}$ defined by $e_i^{\ast} e_j = \delta_{ij}$. I claim that the coefficients $T_{ij}$ give a unique decomposition $T = T_{ij} e_i^{\ast} \otimes f_j$ in $A^{\ast} \otimes B$. The term $e_i^{\ast} \otimes f_j$ is the unique linear operator which sends $e_i$ to $f_j$ and is zero on the rest of the basis.

The isomorphism I just gave doesn't depend on a choice of basis and now one can define nice things like the rank (the minimal number of elements in a decomposition of $T$ into pure tensors) or the trace (the result of sending $e_i^{\ast} \otimes e_i$ to $1$ for a linear operator $T : A \to A$) in a totally basis-free way.

One upshot of everything I just said is that it extends to modules over a ring, which is a much more general situation. Anyway, maybe I'm still not talking in the right direction for you. Could you be more specific about what kind of answer you're looking for?

Answer 6

In differential calculus, you learned that any smooth function can locally be approximated by a linear function. The same principle works for multi-dimensional functions: locally they can be approximated by linear functions. So linear algebra provides tools and structure to grasp multi-dimensional functions at a local level.

That's why so many diverse majors--economics, physics, engineering--tend to require, or at least encourage, students to take a linear algebra course. Such disciplines deal with functions of many dimensions, which are frequently too complex to deal with except at a local level, where they are approximated with a linear function.

Answer 7

Learn that both of your examples can be generalized as tensors.

Answer 8

Computer graphics relies heavily on linear algebra.