real symmetric matrix has real eigenvalues - elementary proof

Question

Every real symmetric matrix has at least one real eigenvalue. Does anyone know how to prove this elementary, that is without the notion of complex numbers?

Answer 1

If "elementary" means not using complex numbers, consider this.

1. First minimize the Rayleigh ratio $R(x)=(x^TAx)/(x^Tx).$ The minimum exists and is real. This is your first eigenvalue.

2. Then you repeat the usual proof by induction in dimension of the space.

3. Alternatively you can consider the minimax or maximin problem with the same Rayleigh ratio, (find the minimum of a restriction on a subspace, then maximum over all subspaces) and it will give you all eigenvalues.

But of course any proof requires some topology. The standard proof requires Fundamental theorem of Algebra, this proof requires existence of a minimum.

Answer 2

How about Jacobi's proof?

See, e.g., Folkmar Bornemann, ``Teacher's Corner - kurze Beweise mit langer Wirkung,'' DMV-Mitteilungen 3-2002, Seite 55 (in German, sorry). Sorry, I don't have the original reference.

The idea is simple, define $\Sigma(A)=\sum_{i=1}^n\sum_{j=i+1}^n a_{ij}^2$ for $A=(a_{ij})$ a symmetric real matrix. Then minimize the function $O(n)\ni J \mapsto \Sigma(J^TAJ)$ over the orthogonal group $O(n)$. The function is continuous and bounded below by zero, and $O(n)$ is compact, so the minimum is attained. But it can not be strictly positive, because if there is an $a_{ij}\not=0$, $i\not=j$, then you can make it zero by a rotation that acts only on the $i$-th and $j$-th row and column, so that it decreases $\Sigma$ (this is a simple little calculation with $2\times 2$ matrices). Therefore the minimum is zero and it is attained in a matrix $J$ for which $J^TAJ$ is diagonal.

The eigenvalues of $A$ are now the (diagonal) entries of $J^TAJ$. No complex numbers are used, but you have to know that the minimum exists. We get the existence of an orthonormal basis consisting of eigenvectors with real eigenvalues.

Answer 3

Let me give it a try. This one only uses the existence of a maximum in a compact set, and the Cauchy-Schwarz inequality.

Let $T$ be a selfadjoint operator in a finite dimensional inner product space.

Claim: $T$ has an eigenvalue $\pm\|T\|$.

Proof: Let $v$ in the unit sphere be such that $\|Tv\|$ attains its maximum value $M=\|T\|$. Let $w$ also in the unit sphere be such that $Mw=Tv$ (which is like saying that $w=\frac{Tv}{\|Tv\|}$, except in the trivial case $T=0$).

This implies that $\langle w,Tv\rangle=M$. In fact, the only way that two unit vectors $v$ and $w$ can satisfy this equation is to have $Tv=Mw$. (Since we know that $\|w\|=1$ and $\|Tv\|\leq M$, the Cauchy-Schwarz inequality tells us that $|\langle w,Tv\rangle|\leq M$, and the equality case is only attainable when $Tv$ is a scalar multiple of $w$, with the scalar $\lambda$ determined by the computation $M=\langle w,Tv\rangle=\langle w,\lambda w\rangle=\lambda\langle w,w\rangle=\lambda$.)

But by selfadjointness of $T$, we also know that $\langle v,Tw\rangle=M$, and this implies, by the same Cauchy-Schwartz-equality reasoning, that $Tw=Mv$.

Now, one of the two vectors $v\pm w$ is nonzero, and we can compute

$T(v\pm w)=Tv\pm Tw=Mw\pm Mv=M(w\pm v)=\pm M(v\pm w)$.

This concludes the proof that $\pm\|T\|$ is eigenvalue with eigenvector $v\pm w$. The reality of the other eigenvalues can be proved by induction, restricting to $(v\pm w)^\bot$ as in the usual proof of the spectral theorem.

Remark: The proof above works with real or complex spaces, and also for compact operators in Hilbert spaces.

Comment: I would like to know if this proof can be found in the literature. I obtained it while trying to simplify a proof of the fact that if $T$ is a bounded selfadjoint operator, then $\|T\|=\sup_{\|v\|\leq 1} \langle Tv,v\rangle$ (as found, for example, on p.32 of Conway J.B., "An Introduction to Functional Analysis"). In the case of non-compact operators, one can only prove that $T$ has as an approximate eigenvalue one of the numbers $\pm\|T\|$. The argument is similar to the one above, but knowledge of the equality case of Cauchy-Schwarz is not enough. One has to know that near-equality implies near-dependence. More precisely, let $v$ be a fixed unit vector, $M\geq 0$ and $\varepsilon\in[0,M]$. If $z$ is a vector with $\|z\|\leq M$ such that $|\langle v,z\rangle|\geq \sqrt{M^2-\epsilon^2}$, then it can be proved that $z$ is within distance $\varepsilon$ of $\langle v,z\rangle v$.

Exercise: Follow the proof (find the possible vectors $v$ and $w$) for the cases in which $T:\mathbb R^2\to\mathbb R^2$ is given by any of the matrices $\begin{pmatrix}2&0\\0&1\end{pmatrix}$, $\begin{pmatrix}-2&0\\0&1\end{pmatrix}$, $\begin{pmatrix}2&0\\0&-2\end{pmatrix}$. This may make clear how the proof was made. Notice that $v$ and $w$ are already eigenvectors in some ("most") cases.

Answer 4

This is just the details of the first step of Alexander Eremenko's answer (so upvote his answer if you like mine), which I think is by far the most elementary. You only need two facts: A continuous function on a compact set in $R^n$ achieves its maximum (or minimum), and the derivative of a smooth function vanishes at a local maximum. And there's no need for Lagrange multipliers at all.

Let $C$ be any closed annulus centered at $0$. The function $$ R(x) = \frac{x\cdot Ax}{x\cdot x}, $$ is continuous on $R^n\backslash\{0\}$ and therefore achieves a maximum on $C$. Since $R$ is homogeneous of degree $0$, any maximum point $x \in C$ is a maximum point on all of $R^n\backslash\{0\}$. Therefore, for any $v \in R^n$, $t = 0$ is a local maximum for the function $$ f(t) = R(x + tv). $$ Differentiating this, we get $$ 0 = f'(0) = \frac{2}{x\cdot x}[Ax - R(x) x]\cdot v $$ This holds for any $v$ and therefore $x$ is an eigenvector of $A$ with eigenvalue $R(x)$.

Answer 5

Another elementary proof, based on the order structure of symmetric matrices. Let me first recall the basic definitions and facts to avoid misunderstandings: we define $A\ge B$ iff $(A-B)x\cdot x\ge0$ for all $x\in\mathbb{R}^n$). Also, a lemma:

A symmetric matrix $A$, which is positive and invertible, is also definite positive (that is, $A\ge\epsilon I$ for some $\epsilon > 0 \,$).

A quick proof passes through the square root of $A$: $(Ax\cdot x)=\|A^{1/2} x\|^2 \ge \|A^{-1/2}\|^{-2} \| x\|^2$; one has to construct $A^{1/2}$ before, without diagonalization, of course.

We may rephrase the lemma saying equivalently: if $A$ is positive but, for any $\epsilon >0$, the matrix $A-\epsilon I$ is not, then $A$ is not invertible.

As a consequence, $\alpha_*:=\inf_{|x|=1}(Ax \cdot x)$ is an eigenvalue of $A$, because $A-\alpha_*I$ is positive and $(A-\alpha_*I)-\epsilon I$ is not (and $\alpha ^ *:=\sup _ {|x|=1}(Ax \cdot x)$ too, for analogous reasons).

The complete diagonalization is then performed inductively, as in other proofs.

Answer 6

This is quite an interesting question, perhaps a research problem. I think an elementary answer should be a high school algebra answer in the sense of Abhyankar and it would have to be in the spirit of what follows. But first a little story.

I was teaching linear algebra and had just covered eigenvalues and characteristic polynomials but was not yet at the chapter on the spectral theorem for real symmetric matrices. I was looking for problems to assign for my students as homework in the textbook we were using. One of the exercises was to show that a real matrix $$ A=\left[ \begin{array}{cc} \alpha & \beta \\\ \beta & \gamma \end{array} \right] $$ only had real eigenvalues. Not too hard. Write the characteristic polynomial $$ \chi(\lambda)=det(\lambda I-A)=\lambda^2-(\alpha+\gamma)\lambda+\alpha\gamma-\beta^2 $$ then its discriminant is $$ \Delta=(\alpha+\gamma)^2-4(\alpha\gamma-\beta^2)=(\alpha-\gamma)^2+4\beta^2\ge 0\ . $$ Hence two real roots.

The next problem in the book was to do the same for $$ A=\left[ \begin{array}{ccc} \alpha & \beta & \gamma\\\ \beta & \delta & \varepsilon \\\ \gamma & \varepsilon & \zeta \end{array} \right] $$ and (silly me) I also assigned it...

Here is the solution in the 3X3 case. All roots are real if the discriminant (for a binary cubic) is nonnegative. The discriminant of the characteristic polynomial is $$ \Delta = (\delta \varepsilon ^{2} + \delta \zeta ^{2} - \zeta \delta ^{2} - \zeta \varepsilon ^{2} + \zeta \alpha ^{2} + \zeta \gamma ^{2} - \alpha \gamma ^{2} - \alpha \zeta ^{2} + \alpha \beta ^{2} + \alpha \delta ^{2} - \delta \alpha ^{2} - \delta \beta ^{2})^{2} \\\ \mbox{} + 14(\delta \gamma \varepsilon - \beta \varepsilon ^{2} + \beta \gamma ^{2} - \alpha \gamma \varepsilon )^{2} \\\ \mbox{} + 2(\delta \alpha \gamma + \delta \beta \varepsilon + \delta \gamma \zeta - \gamma \delta ^{2} - \gamma \varepsilon ^{2} + \gamma ^{3} - \alpha \beta \varepsilon - \alpha \gamma \zeta )^{2} \\\ \mbox{} + 2(\delta \beta \gamma + \delta \varepsilon \zeta - \varepsilon ^{3} + \varepsilon \alpha ^{2} + \varepsilon \gamma ^{2} - \alpha \beta \gamma - \alpha \delta \varepsilon - \alpha \varepsilon \zeta )^{2} \\\ \mbox{} + 2(\zeta \alpha \beta + \zeta \beta \delta + \zeta \gamma \varepsilon - \beta \varepsilon ^{2} - \beta \zeta ^{2} + \beta ^{3} - \delta \alpha \beta - \alpha \gamma \varepsilon )^{2} \\\ \mbox{} + 14(\zeta \beta \varepsilon - \gamma \varepsilon ^{2} + \gamma \beta ^{2} - \alpha \beta \varepsilon )^{2} \\\ \mbox{} + 2(\zeta \beta \gamma + \delta \varepsilon \zeta - \varepsilon ^{3} + \varepsilon \alpha ^{2} + \varepsilon \beta ^{2} - \alpha \beta \gamma - \alpha \delta \varepsilon - \alpha \varepsilon \zeta )^{2} \\\ \mbox{} + 14(\varepsilon \beta ^{2} + \zeta \beta \gamma - \delta \beta \gamma - \varepsilon \gamma ^{2})^{2} \\\ \mbox{} + 2(\zeta \alpha \beta + \zeta \beta \delta + \zeta \gamma \varepsilon - \beta \gamma ^{2} - \beta \zeta ^{2} + \beta ^{3} - \delta \alpha \beta - \delta \gamma \varepsilon )^{2} \\\ \mbox{} + 2(\alpha \gamma \zeta + \zeta \beta \varepsilon - \gamma ^{3} + \gamma \beta ^{2} + \gamma \delta ^{2} - \delta \alpha \gamma - \delta \beta \varepsilon - \delta \gamma \zeta )^{2}\ . $$

This formula comes from a paper by Ilyushechkin in Mat. Zametki, 51, 16-23, 1992.

I suspect the elementary answer should be as follows. First find a list of invariants or covariants of binary forms $C_1,C_2,\ldots$ such that a form with real coefficients has only real roots iff these covariants are nonnegative. Apply this to the characteristic polynomial of a general real symmetric matrix and show that you get sums of squares. I suppose these covariants, via Sturm's sequence type arguments, should correspond to subresultants or rather subdiscriminants. This seems also related to Part 2) of Godsil's answer.

---

Edit: Another recent research reference which relates to the above sum-of-squares formula is the article The entropic discriminant by Sanyal, Sturmfels and Vinzant.

---

Edit 2: I just found out that the problem I mentioned above has been completely solved! See Proposition 4.50 page 127 in the book by Basu, Pollack and Roy on real algebraic geometry. The connection with classical invariants/covariants of binary forms is not apparent but it is there: their proof is based on subresultants and subdiscriminants which are leading terms of $SL_2$ covariants.

Answer 7

Here's one inspired by the Schur-Horn theorem and by Jacobi's proof as described by Uwe Franz: Fix real numbers $a_1>a_2>\cdots>a_n$. For $X$ an $n \times n$ symmetric matrix, define $\psi(X) = \sum_i a_i X_{ii}$.

Let $M$ be the matrix we're trying to diagonalize. Maximize $\psi(J M J^T)$ over $J$ in $SO(n)$. Since $SO(n)$ is compact, $\psi$ has a maximum value; let $X = JMJ^T$ achieve this maximum. For any skew symmetric matrix $Y$, we compute: $$\psi \left( \exp(Y) X \exp(-Y) \right) =\psi \left( X + (YX-XY) + O(|Y|^2) \right) = $$ $$\psi(X) + \sum_{i,j} \left(a_{i} Y_{ij} X_{ji} - a_i X_{ij} Y_{ji} \right) +O(|Y|^2) = \psi(X) + 2 \sum_{i<j} (a_i-a_j) Y_{ij} X_{ij} +O(|Y|^2).$$ (Recall that $X$ is symmetric and $Y$ is skew-symmetric.) So $$\left. \frac{\partial \psi}{\partial Y_{ij}} \right|_{Y=0} = 2 (a_i - a_j) X_{ij}.$$ We see that, at a critical point, all the off diagonal $X_{ij}$ are zero. One can also compute that the Hessian is positive definite only when $X_{11} > X_{22} > \cdots > X_{nn}$. So the maximum occurs at the unique diagonalization for which the eigenvalues appear in order. (If there are repeated eigenvalues, then there is still a unique maximum on the orbit $J M J^T$, but it is achieved by multliple values of $J$, so the Hessian is only positive semi-definite.)

Answer 8

We can do it in two steps.

Step 1: show that if $A$ is a real symmetric matrix, there is an orthogonal matrix $L$ such that $A=LHL^T$, where $H$ is tridiagonal and its off-diagonal entries are non-negative. (Apply Gram-Schmidt to sets of vectors of the form $\{x,Ax,\ldots,A^mx\}$, or use Householder transformations, which is the same thing.)

Step 2. We need to show that the eigenvalues of tridiagonal matrices with non-negative off-diagonal entries are real. We can reduce to the case where $H$ is indecomposable. Assume it is $n\times n$ and let $\phi_{n-r}$ the the characteristic polynomial of the matrix we get by deleting the first $r$ rows and columns of $H$. Then $$ \phi_{n-r+1} = (t-a_r)\phi_{n-r} -b_r \phi_{n-r-1}, $$ where $b>0$. Now prove by induction on $n$ that the zeros of $\phi_{n-r}$ are real and are interlaced by the zeros of $\phi_{n-r-1}$. The key here is to observe that this induction hypothesis is equivalent to the claim that all poles and zeroes of $\phi_{n-r-1}/\phi_{n-r}$ are real, and in its partial fraction expansion all numerators are positive. From this it follows that the derivative of this rational function is negative everywhere it is defined and hence, between each consecutive pair of zeros of $\phi_{n-r-1}$ there must be a real zero of $\phi_{n-r}$.

Answer 9

Essentially the same as Marcos Cossarini's proof, but without Cauchy-Schwarz.

Let $A\in R^{n\times n}$ symmetric of rank $\ge 1$. By compactness let $x\in R^n$ with $\|x\|=1$ be such that $\|Ax \| = \max_{u\in R^n: \|u\|=1} \|Au\|$ and set $y = Ax/\lambda$ for $\lambda=\|Ax\|>0$. Then \begin{align} \|A(x+ y) - \lambda(x+y)\|^2 &=\| Ay - \lambda x\|^2 \\&= \|A y\|^2 + \lambda^2 - 2 \lambda y^TA x \\&= \|A y\|^2 + \lambda^2 - 2\lambda^2 \\&\le 0. \end{align} Either $x+y\ne 0$ is eigenvector for $\lambda$ or $x+Ax/\lambda=0$ and $x$ is is eigenvector for $-\lambda$.

Answer 10

Just found in Godsil-Royle's Algebraic graph theory: One first proves that two eigenvectors associated with two different eigenvalues are necessarily orthogonal to each other (pretty standard), then observes that if $u$ is eigenvector associated with eigenvalue $\lambda$, then $\bar u$ is eigenvector associated with eigenvalue $\bar\lambda$. Now the eigenvalues $\lambda,\bar\lambda$ cannot be different, for otherwise by the above observation $0=u^T u=\|u\|^2$ although $u\not=0$.

(It does contain complex numbers, but is still amazingly straightforward).

Answer 11

My solution is based in the same idea as the currently accepted answer. I am also convinced that it is necessary to use analysis (and topology), as for any proof of the Fundamental Theorem of Algebra. However, IMO, it is easier to prove the orthogonal diagonalisation of real symmetric matrices than to prove the Fundamental Theorem of Algebra.

Instead of a symmetric matrix $A$, I will talk about a symmetric operator $h$ on a finite-dimensional real vector space $E$ with a scalar product $\langle\cdot,\cdot\rangle$. (The matrix $A$ can be thought of as the matrix of $h$ with respect to some orthonormal base.)

The goal is to prove $h$ has an eigenvector in $E$. This follows from the following lemma:

Lemma. Let $\boldsymbol{u}\in E$ be such that:

1. $\Vert\boldsymbol{u}\Vert = 1$,
2. $\langle{\boldsymbol{u}},\,{h(\boldsymbol{u})}\rangle\ge\langle{\boldsymbol{x}},\,{h(\boldsymbol{x})}\rangle$ for all $\boldsymbol{x}\in E$ such that $\Vert \boldsymbol{x}\Vert = 1$.

Then:

1. $\langle{\boldsymbol{u}},\,{h(\boldsymbol{x})}\rangle =\langle{\boldsymbol{x}},\,{h(\boldsymbol{u})}\rangle = 0$ for all $\boldsymbol{x}\in E$ such that $\boldsymbol{x}\perp \boldsymbol{u}$,
2. the orthogonal subspace $\boldsymbol{u}^\perp$ of $\boldsymbol{u}$ is invariant by $h$,
3. the subspace spanned by $\boldsymbol{u}$ is invariant by $h$.

Indeed, the set of $\boldsymbol{x}\in E$ such that $\Vert \boldsymbol{x}\Vert = 1$ is compact, therefore, the values of $\langle{\boldsymbol{x}},\,{h(\boldsymbol{x})}\rangle$ for $\boldsymbol{x}\in E$ such that $\Vert \boldsymbol{x}\Vert = 1$ have a maximum. Let $\boldsymbol{u}$ be the value of $\boldsymbol{x}$ at which the maximum is reached. Then $\boldsymbol{u}$ satisfies the hypotheses of the lemma and, according to the lemma, it is an eigenvector of $h$.

Proof of the lemma. Let $g\colon E\to\mathbf{R}$ be defined by: $$ g(\boldsymbol{x}) =\langle{\boldsymbol{x}},\,{h(\boldsymbol{x})}\rangle \quad\text{for all}\quad \boldsymbol{x}\in E. $$

Take an arbitrary $\boldsymbol{v}\in E$ be such that $\boldsymbol{v}\perp \boldsymbol{u}$. Let $f\colon\mathbf{R}\to E$ be defined by: $$ f(t) =\frac{\boldsymbol{u} + t\boldsymbol{v}}{\Vert{\boldsymbol{u} + t\boldsymbol{v}}\Vert}. $$ Then, $f(0) = \boldsymbol{u}$ and $\Vert{f(t)}\Vert = 1$ for all $t\in\mathbf{R}$. Thus, $(g\circ f)$ atteins a global maximum at $0$. Therefore, $(g\circ f)'(0) = 0$ if $(g\circ f)'(0)$ exists. We have: \begin{align*} (g\circ f)(t) &=g(f(t)) =\langle{f(t)},\,{h(f(t))}\rangle =\frac{\langle{\boldsymbol{u} + t\boldsymbol{v}},\,{h(\boldsymbol{u} + t\boldsymbol{v})}\rangle}{\Vert{\boldsymbol{u} + t\boldsymbol{v}}\Vert^2}\\ &=\frac{\langle{\boldsymbol{u} + t\boldsymbol{v}},\,{h(\boldsymbol{u} + t\boldsymbol{v})}\rangle}{\langle{\boldsymbol{u} + t\boldsymbol{v}},\,{\boldsymbol{u} + t\boldsymbol{v}}\rangle}\\ &=\frac{ \langle{\boldsymbol{u}},\,{h(\boldsymbol{u})}\rangle + t\langle{\boldsymbol{u}},\,{h(\boldsymbol{v})}\rangle + t\langle{\boldsymbol{v}},\,{h(\boldsymbol{u})}\rangle + t^2\langle{\boldsymbol{v}},\,{h(\boldsymbol{v})}\rangle }{\langle{\boldsymbol{u}},\,{\boldsymbol{u}}\rangle + t^2\langle{\boldsymbol{v}},\,{\boldsymbol{v}}\rangle}\\ &=\frac{ \langle{\boldsymbol{u}},\,{h(\boldsymbol{x})}\rangle + t(\langle{\boldsymbol{u}},\,{h(\boldsymbol{v})}\rangle +\langle{\boldsymbol{v}},\,{h(\boldsymbol{u})}\rangle) + t^2\langle{\boldsymbol{v}},\,{h(\boldsymbol{v})}\rangle }{1 + t^2\Vert \boldsymbol{v}\Vert^2}. \end{align*} A usual calculation shows that $$ (g\circ f)'(0) =\langle{\boldsymbol{u}},\,{h(\boldsymbol{v})}\rangle +\langle{\boldsymbol{v}},\,{h(\boldsymbol{u})}\rangle = 2\langle{\boldsymbol{u}},\,{h(\boldsymbol{v})}\rangle = 2\langle{\boldsymbol{v}},\,{h(\boldsymbol{u})}\rangle. $$ Therefore, $\langle{\boldsymbol{u}},\,{h(\boldsymbol{v})}\rangle =\langle{\boldsymbol{v}},\,{h(\boldsymbol{u})}\rangle = 0$.

The first part of the conclusion of the lemma is proved. The other two follow easily. $\square$