# Differentiability of Eigenvalues - Perturbation Theory

first, I have a general question. In perturbation theory, I saw perturbations in eigenvalues and eigenvectors of square, non-symmetric matrices and the calculations were all right but no one ever proved the differentiability of the eigenvalues/eigenvectors in the perturbation-parameter! They just assumed it. So, why is that?

Second, does someone know one way to prove differentiability (at least twice) of eigenvalues/eigenvectors in the perturbation-parameter for those matrices in the perturbation-parameter?

• Do you have a specific reference you are using? Differentiability should depend on the perturbation being used, as the map from coefficients of a polynomial to its solutions is not differentiable everywhere. – user44191 May 11 '18 at 18:21
• Check Tosio Kato's book, Perturbation Theory for Linear Operators – Pietro Majer May 11 '18 at 19:52
• I read Kato's book also but unfortunately I couldn't find the case of a square, non-symmetric, real-valued matrix with at least one simple eigenvalue. – xmonetx May 11 '18 at 20:09

The complete reference is Kato's book Perturbation theory .... But perhaps you need only the most basic results. Then see my book Matrices (Springer GTM #216), 2nd edition. This is Section 5.2.

Mind however that this is a much more elaborate topic than what you could think at first glance. There are at least four completely distinct aspects:

• On the one hand, the spectrum is always a continuous function of the entries. But this continuity is false in full generality for individual eigenvalues.
• Next, if an eigenvalue $\lambda_0$ of a given matrix $M_0$ is algebraically simple, then there exists locally an analytic function $M\longmapsto\lambda(M)$ where $\lambda(M)$ is an eigenvalue of $M$ and $\lambda(M_0)=\lambda_0$. This is the case where you can compute by hand the derivatives of an eigenvalue.
• Finally, if $t\mapsto H(t)$ is a one-parameter family (a curve) of Hermitian matrices (think to real symmetric matrices), then the eigenvalues of $H(t)$ can be arranged as a list of functions $t\mapsto\lambda_j(t)$ which inherit the regularity ($C^k$, analyticity) of the data. Mind that there are crossings, the list $\vec\lambda(t)$ is not ordered. The statement becomes deadly false for a two-parameter family (a surface).
• However, the ordered list $\lambda_1(H)\le\cdots\le\lambda_n(H)$ is Lipschitz continuous over the space of Hermitian matrices, with Lipschitz constant one: $$|\lambda_j(K)-\lambda_j(H)|\le\|K-H\|$$ where the norm is the operator norm (equal to the spectral radius for Hermitian matrices).
• Thank you very much, I'll read your book to get more details. – xmonetx May 11 '18 at 20:16
• Would you please explain your second bullet point in greater detail? The relation between $M$ and $M_0$ and the statement about the existence of an analytic function regarding an simple eigenvalue, is there any theorem to that? – xmonetx May 14 '18 at 9:21
• @xmonetx The adjective "locally" means that the function $\lambda$ is defined and analytic for $M$ in some neighbourhood of $M_0$. The proof is a simple application of Implicit Function Theorem. – Denis Serre May 14 '18 at 10:04
• Hi, your Hi, for my research I am busy with symmetric matrices. Now, your 3rd bullet means that simple eigenvalues of a one-parameter differentiable family of symmetric matrices H(t) are differentiable. You say that this does not hold if you have more than one-parameter. Why is that and can you give an example, as I do not see the problem in fact? – Koen May 26 '18 at 15:51
• @Koen. I did not write simple. The eigenvalues may cross. I wrote one parameter. If you have more than one, the differentiability fails. Just consider the two parameter family $\begin{pmatrix} s & t \\ t & -s \end{pmatrix}$, whose eigenvalues are $\pm\sqrt{s^2+t^2}$. They cannot be arranged so as to be differentiable at the origin. – Denis Serre May 27 '18 at 6:57

Eigenvalues are roots of the characteristic polynomial. In the case the polynomial has simple roots, they smoothly depend on the coefficients. Below is the proof in the real case. The argument below can be easily modified to more general situations.

Theorem. For $a=(a_0,a_1,\ldots,a_n)\in\mathbb{R}^{n+1}$, $a_n\neq 0$ let $P_a(x)=a_nx^n+\ldots+a_1x+a_0$. Suppose that for $a^0=(a_0^0,\ldots,a_n^0)$, $a^0_n\neq 0$ the polynomial $P_{a^0}(x)$ has $n$ distinct real roots. Then, there is $\epsilon>0$ and $C^\infty$ smooth functions $$\lambda_1,\ldots,\lambda_n: B^{n+1}(a^0,\epsilon)\to\mathbb{R}$$ such that for any $a\in B^{n+1}(a^0,\epsilon)$, $\lambda_1(a),\ldots,\lambda_n(a)$ are distinct roots of the polynomial $P_a(x)$. In other words, prove that in a small neighborhood of $a^0$, roots of the polynomial $P_a$ depend smoothly on the coefficients $a_0, a_1,\ldots, a_n$.

Proof. Denote the roots of $P_{a^0}$ by $\lambda_1^0,\ldots,\lambda_n^0$. That is $P_{a^0}(\lambda)=a_n^0(\lambda-\lambda_1^0)\cdots(\lambda-\lambda_n^0)$ and clearly $$\left.\frac{d}{d\lambda}\right|_{\lambda=\lambda_i^0} P_{a^0}(\lambda)\neq 0 \quad \mbox{for all i=1,2,\ldots,n.}$$ This is where we employ the assumption that the roots are distinct.

Let $F(a,\lambda)=F(a_0,a_1,\ldots,a_n,\lambda)=P_a(\lambda)$. For each $i=1,2,\ldots,n$ we have $$F(a_0^0,a_1^0,\ldots,a_n^0,\lambda_i^0)=P_{a^0}(\lambda_i^0)=0$$ and $$\left.\frac{\partial}{\partial\lambda}\right|_{\lambda=\lambda_i^0} F(a_0^0,a_1^0,\ldots a_n^0,\lambda) = \left.\frac{d}{d\lambda}\right|_{\lambda=\lambda_i^0} P_{a^0}(\lambda)\neq 0.$$ Thus according to the Implicit Function Theorem, there is a unique $C^\infty$ smooth function $\lambda(a)$ defined for $a=(a_0,a_1,\ldots,a_n)$ in a neighborhood of $a^0=(a_0^0,a_1^0,\ldots,a_n^0)$ (neighborhood in $\mathbb{R}^{n+1}$) such that $\lambda(a^0)=\lambda^0_i$ and $P_a(\lambda(a))=F(a,\lambda(a))=0$. Denote this function by $\lambda_i(a)$. Hence $\lambda_i(a)$, $i=1,2,\ldots,n$ are roots of the polynomial $P_a$. Since $\lambda_i(a^0)\neq \lambda_j(a^0)$ for $i\neq j$, we see that these roots are distinct in a neighborhood $B^{n+1}(a^0,\epsilon)$ of $a^0$, provided $\epsilon>0$ is sufficiently small. $\Box$

• @xmonetx My argument works if one eigenvalue is simple and you have no idea what happens to the others. Then it smoothly depends on the coefficients of the characteristic polynomial. Just read the proof and adapt it to the situation you have. – Piotr Hajlasz May 11 '18 at 20:40
• All right, thank you very much for your effort. – xmonetx May 11 '18 at 21:05

The eigenvalues of a square matrix $A$ are the roots of the characteristic polynomial, and are analytic except where their multiplicities change. Thus if (in a certain open region of parameter space) there is one eigenvalue, of algebraic multiplicity $m$, inside a positively oriented closed contour $\Gamma$ in $\mathbb C$, and no eigenvalues on $\Gamma$, that eigenvalue is

$$\frac{1}{2\pi im}\oint_\Gamma \frac{z P'(z)}{P(z)}\; dz$$ where $P(z) = \det(A - z I)$, and this is analytic in the entries of $A$. The corresponding eigenvectors (with an appropriate normalization) can be found by solving a linear system depending linearly on the parameters and the eigenvalue, and these are also analytic.