Derivative of eigenvalues of a symmetric tridiagonal matrix built via the Lanczos-Arnoldi scheme

Question

Suppose $\mathbf{A}(\mu)$ being a symmetric positive definite matrix of dimension $n$ where its elements depend parametrically on the real parameter $\mu$.

Suppose now to build the orthonormal basis of the Krylov subspace from an initial normalized guess $\mathbf{x}_0$ by performing $m$ iterations of the Lanczos-Arnoldi algorithm. This orthonormal basis is gathered in the unitary matrix $\mathbf{V}$, dimension ($n,m$), and transforms $\mathbf{A}$ into a tridiagonal symmetric matrix $\mathbf{T}$ of dimension $m$ which can be finally diagonalized by a unitary transformation $\mathbf{U}$ as shown below where $\boldsymbol{\Lambda}$ denotes the final diagonal matrix. \begin{equation} \begin{split} \mathbf{T}&=\mathbf{V}^T\mathbf{AV}\\ \boldsymbol{\Lambda}&=\mathbf{U}^T\mathbf{TU}\end{split} \end{equation} We are now interested in the derivative of $\boldsymbol{\Lambda}$ with respect to the parameter $\mu$: this very problem was already discussed in a previous question (Derivative of eigenvectors of an Hermitian matrix) and the result is reported below. \begin{equation} \frac{d\boldsymbol{\Lambda}}{d\mu}=\mathbf{U}^T\frac{d\mathbf{T}}{d\mu}\mathbf{U} \end{equation} In our case, however, $\mathbf{T}$ is obtained via the trasfomation $\mathbf{V}$ which is the orthonormal basis of the Krylov subspace mentioned.

The question is therefore, can I compute the derivative of $\mathbf{T}$ by simply differentiating $\mathbf{A}(\mu)$ (equation below), i.e. similarly to the case for $\boldsymbol{\Lambda}$ where we ignored the derivatives of the transformation matrix, or should I consider differentiating $\mathbf{V}$ as well? \begin{equation} \frac{d\mathbf{T}}{d\mu}=\mathbf{V}^T\frac{d\mathbf{A}}{d\mu}\mathbf{V} \end{equation}

Answer 1

The magic of "not having to differentiate the eigenvectors" is known as the Hellmann–Feynman theorem. Let me walk you through it, at the level of a single eigenvalue, to answer the question you asked in the comment.

You decompose the real symmetric matrix $A$ as $A=W\Lambda W^T$, with $W$ the orthogonal matrix of eigenvectors and $\Lambda={\rm diag}\,(\lambda_1,\lambda_2,\ldots)$ the diagonal matrix of eigenvalues. Consider one eigenvalue $\lambda_k$ and the associated eigenvector $\psi$ with elements $\psi_i=W_{ik}$.
By construction, the eigenvalue equals the inner product $$\lambda_k=(\psi , A\psi)=(A\psi , \psi),$$ because $(\psi,\psi)=1$. Now take the derivative with respect to $\mu$, denoted by a prime: $$\lambda'_k=(\psi',A\psi)+(A\psi,\psi')+(\psi,A'\psi)=$$ $$\qquad=\lambda_k(\psi',\psi)+\lambda_k(\psi,\psi')+(\psi,A'\psi)$$ $$\qquad=\lambda_k\frac{d}{d\mu}(\psi,\psi)+(\psi,A'\psi)$$ $$\qquad=0+(\psi,A'\psi).$$ So you see, the derivative of the wave functions drops out because of the normalization.