# Iterative method to find the derivative of eigenvectors of a large matrix

Given a large real symmetric matrix, $$\mathbf{A(m)}\in\mathbb{R}^{N\times N}$$, which elements depend on a parameters vector, $$\mathbf{m}\in\mathbb{R}^m$$. The matrix elements cannot be stored explicitly, but we can compute the matrix-vector operation, $$\mathbf{A(m)v}$$.

Let's say we have obtained $$B$$ ($$B\ll N$$) lowest eigenvectors and eigenvalues of $$\mathbf{A(m)}$$ as $$\mathbf{V_B}$$ and $$\mathbf{\Lambda_B}$$, respectively. The matrix of eigenvectors and diagonal matrix of eigenvalues satisfy $$\mathbf{A(m)V_B = V_B\Lambda_B}.$$ Is there any iterative or direct method to calculate the derivative of a defined loss function with respect to every elements of $$\mathbf{m}$$, i.e. $$\partial \mathcal{L}/\partial \mathbf{m}$$ given $$\partial \mathcal{L}/\partial \mathbf{V_B}$$, $$\partial \mathcal{L}/\partial \mathbf{\Lambda_B}$$, $$\mathbf{V_B}$$, $$\mathbf{\Lambda_B}$$? The operation $$\frac{\partial \mathbf{A}}{\partial m_i}\mathbf{v}$$ can also be computed.

Please note that the matrix $$\mathbf{A}$$ is too large to be stored in memory, so I want to avoid storing all the eigenvectors (but storing $$\mathbf{V_B}$$ is fine).

For a case where $$\partial \mathcal{L}/\partial \mathbf{V_B} = \mathbf{0}$$, I can solve the problem. From perturbation theory, we perturbation of the $$b$$-th eigenvalue, $$\lambda_b$$, is given by $$\delta\lambda_b = \mathbf{v}^T_\mathbf{b}\mathbf{\delta A}\mathbf{v_b}.$$ I assume $$\mathbf{\delta A}$$ is symmetric to preserve the symmetricity of the matrix $$\mathbf{A}$$. Then, the derivative of a loss function with respect to every element in $$\mathbf{A}$$ is given by $$\frac{\partial \mathcal{L}}{\partial \mathbf{A}}=\sum_{b}\mathbf{v_b}\mathbf{v}_{\mathbf{b}}^T \frac{\partial \mathcal{L}}{\partial \lambda_b} = \mathbf{V_B}\frac{\partial \mathcal{L}}{\partial \mathbf{\Lambda_B}}\mathbf{V}^T_\mathbf{B}.$$

Using the chain rule, we obtain $$\frac{\partial \mathcal{L}}{\partial m_i} = \sum_{j,k}\frac{\partial\mathcal{L}}{\partial \mathbf{A}_{jk}} \frac{\partial \mathbf{A}_{jk}}{\partial m_i}=\mathrm{tr}\left[\mathbf{V}^T_\mathbf{B}\frac{\partial \mathcal{L}}{\partial \mathbf{\Lambda_B}}\frac{\partial\mathbf{A}}{\partial m_i}\mathbf{V_B}\right]$$

The problem is, if $$\partial \mathcal{L}/\partial \mathbf{V_B} \ne \mathbf{0}$$, I cannot find the expression that does not involve the full matrix $$\mathbf{A}$$ or the full eigenvalues matrix.