Differentiability of Eigenvalues - Perturbation Theory

Question

first, I have a general question. In perturbation theory, I saw perturbations in eigenvalues and eigenvectors of square, non-symmetric matrices and the calculations were all right but no one ever proved the differentiability of the eigenvalues/eigenvectors in the perturbation-parameter! They just assumed it. So, why is that?

Second, does someone know one way to prove differentiability (at least twice) of eigenvalues/eigenvectors in the perturbation-parameter for those matrices in the perturbation-parameter?

Answer 1

The complete reference is Kato's book Perturbation theory .... But perhaps you need only the most basic results. Then see my book Matrices (Springer GTM #216), 2nd edition. This is Section 5.2.

Mind however that this is a much more elaborate topic than what you could think at first glance. There are at least four completely distinct aspects:

• On the one hand, the spectrum is always a continuous function of the entries. But this continuity is false in full generality for individual eigenvalues.
• Next, if an eigenvalue $\lambda_0$ of a given matrix $M_0$ is algebraically simple, then there exists locally an analytic function $M\longmapsto\lambda(M)$ where $\lambda(M)$ is an eigenvalue of $M$ and $\lambda(M_0)=\lambda_0$. This is the case where you can compute by hand the derivatives of an eigenvalue.
• Finally, if $t\mapsto H(t)$ is a one-parameter family (a curve) of Hermitian matrices (think to real symmetric matrices), then the eigenvalues of $H(t)$ can be arranged as a list of functions $t\mapsto\lambda_j(t)$ which inherit the regularity ($C^k$, analyticity) of the data. Mind that there are crossings, the list $\vec\lambda(t)$ is not ordered. The statement becomes deadly false for a two-parameter family (a surface).
• However, the ordered list $\lambda_1(H)\le\cdots\le\lambda_n(H)$ is Lipschitz continuous over the space of Hermitian matrices, with Lipschitz constant one: $$|\lambda_j(K)-\lambda_j(H)|\le\|K-H\|$$ where the norm is the operator norm (equal to the spectral radius for Hermitian matrices).

Answer 2

Eigenvalues are roots of the characteristic polynomial. In the case the polynomial has simple roots, they smoothly depend on the coefficients. Below is the proof in the real case. The argument below can be easily modified to more general situations.

Theorem. For $a=(a_0,a_1,\ldots,a_n)\in\mathbb{R}^{n+1}$, $a_n\neq 0$ let $P_a(x)=a_nx^n+\ldots+a_1x+a_0$. Suppose that for $a^0=(a_0^0,\ldots,a_n^0)$, $a^0_n\neq 0$ the polynomial $P_{a^0}(x)$ has $n$ distinct real roots. Then, there is $\epsilon>0$ and $C^\infty$ smooth functions $$ \lambda_1,\ldots,\lambda_n: B^{n+1}(a^0,\epsilon)\to\mathbb{R} $$ such that for any $a\in B^{n+1}(a^0,\epsilon)$, $\lambda_1(a),\ldots,\lambda_n(a)$ are distinct roots of the polynomial $P_a(x)$. In other words, prove that in a small neighborhood of $a^0$, roots of the polynomial $P_a$ depend smoothly on the coefficients $a_0, a_1,\ldots, a_n$.

Proof. Denote the roots of $P_{a^0}$ by $\lambda_1^0,\ldots,\lambda_n^0$. That is $P_{a^0}(\lambda)=a_n^0(\lambda-\lambda_1^0)\cdots(\lambda-\lambda_n^0)$ and clearly $$ \left.\frac{d}{d\lambda}\right|_{\lambda=\lambda_i^0} P_{a^0}(\lambda)\neq 0 \quad \mbox{for all $i=1,2,\ldots,n$.} $$ This is where we employ the assumption that the roots are distinct.

Let $F(a,\lambda)=F(a_0,a_1,\ldots,a_n,\lambda)=P_a(\lambda)$. For each $i=1,2,\ldots,n$ we have $$ F(a_0^0,a_1^0,\ldots,a_n^0,\lambda_i^0)=P_{a^0}(\lambda_i^0)=0 $$ and $$ \left.\frac{\partial}{\partial\lambda}\right|_{\lambda=\lambda_i^0} F(a_0^0,a_1^0,\ldots a_n^0,\lambda) = \left.\frac{d}{d\lambda}\right|_{\lambda=\lambda_i^0} P_{a^0}(\lambda)\neq 0. $$ Thus according to the Implicit Function Theorem, there is a unique $C^\infty$ smooth function $\lambda(a)$ defined for $a=(a_0,a_1,\ldots,a_n)$ in a neighborhood of $a^0=(a_0^0,a_1^0,\ldots,a_n^0)$ (neighborhood in $\mathbb{R}^{n+1}$) such that $\lambda(a^0)=\lambda^0_i$ and $P_a(\lambda(a))=F(a,\lambda(a))=0$. Denote this function by $\lambda_i(a)$. Hence $\lambda_i(a)$, $i=1,2,\ldots,n$ are roots of the polynomial $P_a$. Since $\lambda_i(a^0)\neq \lambda_j(a^0)$ for $i\neq j$, we see that these roots are distinct in a neighborhood $B^{n+1}(a^0,\epsilon)$ of $a^0$, provided $\epsilon>0$ is sufficiently small. $\Box$

Answer 3

The eigenvalues of a square matrix $A$ are the roots of the characteristic polynomial, and are analytic except where their multiplicities change. Thus if (in a certain open region of parameter space) there is one eigenvalue, of algebraic multiplicity $m$, inside a positively oriented closed contour $\Gamma$ in $\mathbb C$, and no eigenvalues on $\Gamma$, that eigenvalue is

$$ \frac{1}{2\pi im}\oint_\Gamma \frac{z P'(z)}{P(z)}\; dz$$ where $P(z) = \det(A - z I)$, and this is analytic in the entries of $A$. The corresponding eigenvectors (with an appropriate normalization) can be found by solving a linear system depending linearly on the parameters and the eigenvalue, and these are also analytic.