Given real symmetric matrix $\mathbf{M}$ with eigenvalues $\lambda_i$ and eigenvectors $\mathbf{v}_i$, the derivative of an eigenvector is $$\dot{\mathbf{v}}_i = \sum_{j \ne i} \frac{\mathbf{v}_j \mathbf{v}_j^T}{\lambda_i - \lambda_j} \dot{\mathbf{M}} \mathbf{v}_i$$

This is obviously not defined when $\lambda_i$ is degenerate. However, even if $\mathbf{M}$ contains degenerate eigenvalues, it is still possible to evaluate $\dot{\mathbf{v}}_i$ so long as $\lambda_i$ is not itself degenerate.

My question is whether it is possible in this case to evaluate the second derivative, $\ddot{\mathbf{v}}_i$. Applying the chain rule to the above expression for $\dot{\mathbf{v}}_i$, I obtain $$\ddot{\mathbf{v}}_i = \sum_{j \ne i} \left[ \frac{\dot{\mathbf{v}}_j \mathbf{v}_j^T + \mathbf{v}_j \dot{\mathbf{v}}_j^T}{\lambda_i - \lambda_j} \dot{\mathbf{M}} \mathbf{v}_i - \frac{\left(\dot{\lambda}_i - \dot{\lambda}_j\right)\mathbf{v}_j \mathbf{v}_j^T}{\left(\lambda_i - \lambda_j\right)^2} \dot{\mathbf{M}} \mathbf{v}_i + \frac{\mathbf{v}_j \mathbf{v}_j^T}{\lambda_i - \lambda_j} \left(\ddot{\mathbf{M}} \mathbf{v}_i + \dot{\mathbf{M}} \dot{\mathbf{v}}_i\right)\right]$$

This suggests that $\ddot{\mathbf{v}}_i$ is undefined if $\mathbf{M}$ has any degenerate eigenvalues. However, finite difference testing seem to suggest that $\ddot{\mathbf{v}}_i$ is defined so long as $\lambda_i$ is not degenerate.

Is $\ddot{\mathbf{v}}_i$ actually defined in this case? If so, is there an analytical expression for it?

A related side-question: in cases where $\ddot{\mathbf{v}}_i$ is unambiguously defined, is it necessarily orthogonal to $\mathbf{v}_i$? I understand the reasoning behind $\dot{\mathbf{v}}^T \mathbf{v} = 0$, but I'm unsure whether the same logic holds when it comes to the second derivative.


If $\lambda_i$ is simple, then the eigenvalue and eigenvector are as smooth as your matrix will allow. You start from $(M-\lambda_i)v_i=0$, $v_i^Tv_i=1$. Differentiating this once yields $(M-\lambda_i)\dot v_i+(\dot M-\dot\lambda_i)v_i=0$, $\dot v_i^Tv_i=0$. From this you calculate $\dot\lambda_i=v_i^T\dot Mv_i$, and the expression for $\dot v_i$ which you gave. Now take another derivative to obtain $(M-\lambda_i)\ddot v_i+2(\dot M-\dot\lambda_i)\dot v_i+(\ddot M-\ddot\lambda_i)v_i=0$, $\ddot v_i^Tv_i+\dot v_i^T\dot v_i=0$. You know everything except $\ddot v_i$ and $\ddot\lambda_i$. If you multiply the first equation with $v_i^T$, you obtain $\ddot\lambda_i$, and subsequently you can find $\ddot v_i$. If $M$ is smooth enough, you can continue the same procedure indefinitely to obtain derivatives of any order. If $M$ is not symmetric, you can do something similar, but you need the eigenvector of the adjoint.

  • $\begingroup$ Thank you, this really helped me re-frame the problem and find the answer I was looking for. I was throwing away the intermediate result $\dot{\mathbf{v}}_i^T \mathbf{v}_i = 0$ and not considering it in the derivation of $\ddot{\mathbf{v}}_i$, which I was incorrectly calculating to be orthogonal to $\mathbf{v}_i$. I somehow suspected I was missing something along those lines, hence the side-question at the end of my post. I am now getting good agreement between numerical differentiation and the analytical result. $\endgroup$
    – ehermes
    Oct 22 '20 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.