Given real symmetric matrix $\mathbf{M}$ with eigenvalues $\lambda_i$ and eigenvectors $\mathbf{v}_i$, the derivative of an eigenvector is $$\dot{\mathbf{v}}_i = \sum_{j \ne i} \frac{\mathbf{v}_j \mathbf{v}_j^T}{\lambda_i - \lambda_j} \dot{\mathbf{M}} \mathbf{v}_i$$

This is obviously not defined when $\lambda_i$ is degenerate. However, even if $\mathbf{M}$ contains degenerate eigenvalues, it is still possible to evaluate $\dot{\mathbf{v}}_i$ so long as $\lambda_i$ is not itself degenerate.

My question is whether it is possible in this case to evaluate the second derivative, $\ddot{\mathbf{v}}_i$. Applying the chain rule to the above expression for $\dot{\mathbf{v}}_i$, I obtain $$\ddot{\mathbf{v}}_i = \sum_{j \ne i} \left[ \frac{\dot{\mathbf{v}}_j \mathbf{v}_j^T + \mathbf{v}_j \dot{\mathbf{v}}_j^T}{\lambda_i - \lambda_j} \dot{\mathbf{M}} \mathbf{v}_i - \frac{\left(\dot{\lambda}_i - \dot{\lambda}_j\right)\mathbf{v}_j \mathbf{v}_j^T}{\left(\lambda_i - \lambda_j\right)^2} \dot{\mathbf{M}} \mathbf{v}_i + \frac{\mathbf{v}_j \mathbf{v}_j^T}{\lambda_i - \lambda_j} \left(\ddot{\mathbf{M}} \mathbf{v}_i + \dot{\mathbf{M}} \dot{\mathbf{v}}_i\right)\right]$$

This suggests that $\ddot{\mathbf{v}}_i$ is undefined if $\mathbf{M}$ has *any* degenerate eigenvalues. However, finite difference testing seem to suggest that $\ddot{\mathbf{v}}_i$ is defined so long as $\lambda_i$ is not degenerate.

Is $\ddot{\mathbf{v}}_i$ actually defined in this case? If so, is there an analytical expression for it?

A related side-question: in cases where $\ddot{\mathbf{v}}_i$ is unambiguously defined, is it necessarily orthogonal to $\mathbf{v}_i$? I understand the reasoning behind $\dot{\mathbf{v}}^T \mathbf{v} = 0$, but I'm unsure whether the same logic holds when it comes to the second derivative.