Nearby matrices have nearby leading eigenvectors?

Question

Suppose I have a symmetric positive semidefinite matrix $A$ with leading eigenvalue $1$ of multiplicity $1$ and remaining eigenvalues $\leq\epsilon$. I am told that another symmetric positive semidefinite matrix $B$ is close in some sense to $A$. I wish to conclude that the leading eigenvector of $B$ is therefore close in some sense to that of $A$.

Question: What notions of closeness correspond to existing theorems of this sort?

One solution: Let $a$ and $b$ denote leading eigenvectors of $d\times d$ matrices $A$ and $B$, respectively, scaled so that $\|a\|_2^2=1$ and $\|b\|_2^2$ is the leading eigenvalue of $B$. Then by triangle, Eckart–Young–Mirsky, and triangle again, we have

$$\|bb^\top-aa^\top\|_F\leq\|bb^\top-B\|_F+\|B-aa^\top\|_F\leq2\|B-aa^\top\|_F\leq2\big(\|B-A\|_F+\|A-aa^\top\|_F\big)\leq2\|B-A\|_F+2\sqrt{d-1}\epsilon.$$

For this attempt, I don't like the additive loss in $\epsilon$, and I doubt it's necessary. Can I get better performance with a different norm? Perhaps the spectral norm?

Answer 1

You can get a nice estimate using the spectral norm.

Theorem 3.2.32 of Bratteli and Robinson vol. 1 tell us an estimate on $f(A) - f(B)$ for a variety of functions. Take $f:\mathbb R \rightarrow \mathbb R$ to be zero on $(-\infty, \epsilon]$ and one on $[1,\infty)$, interpolating with some quadratics. I think you get $$ \|f(A) - f(B)\| \leq \frac{2\|A - B\|}{1-\epsilon} $$ and this helps. Notice $f(A) = aa^*$ and $f(B) = bb^*$ and I assume you used transpose because you assumed real scalars. Works either way, I think.

I should add I am not entirely sure of the constant, just that the bound in linear in $\|A - B\|$.

Answer 2

The spectral projection of $A$ for eigenvalue $1$ can be realized as $P_A = \dfrac{1}{2\pi i} \oint_{\Gamma} (z I - A)^{-1} \; dz$ where $\Gamma$ is a contour that encloses $1$ but none of the other eigenvalues. In particular, take $\Gamma$ to be a circle of radius $r$ centred at $1$, where $r = (1-\epsilon)/2$. For $z \in \Gamma$ we have (using the $\ell^2$ operator norm) $\|(z I - A)^{-1}\| \le 1/r$. Then if $\|B - A\| < r$, $z I - B$ is invertible for $z \in \Gamma$, with $$\|(z I - B)^{-1} - (zI-A)^{-1} \| \le \dfrac{\|(z I - A)^{-1}\|^2 \|B-A\|}{1 - \|B - A\| \|(zI - A)^{-1}\|} \le \dfrac{\|B-A\|}{r(r - \|B-A\|)}$$ and $P_B = \dfrac{1}{2\pi i} \oint_{\Gamma} (z I - B)^{-1} \; dz$ is a spectral projection of rank $1$ with $\|P_B - P_A\| \le \dfrac{\|B-A\|}{r - \|B-A\|}$.