# Invariant subspaces for compact restrictions

Suppose $Y$ is a closed hyperplane in $X$, so we can write $X=Y\oplus[x_0]$. Let $y_n$ be a normalized basis of $Y$. Define an operator $S:Y\to Y\oplus[x_0]$ by $Sy_n=\alpha_ny_n+\beta_nx_0$, for any $n=1,2,\dots$. We can choose $\alpha_n\to 0$ and $\beta_n\to 0$ such that $S$ is compact. Can we find a non-trivial subspace $Z$ of $Y$ such that $S(Z)\subseteq Z$?

Edit: I previously posted a more general question, but I just realized it has a negative answer. This is the concrete example I have.

-
I don't understand, what if $S(Y) \cap Y =\{0\}$? –  Kevin Beanland Apr 26 '12 at 16:59
Yes, the initial question had a negative answer for many reasons. Hopefully the above makes more sense. –  Mathy Apr 26 '12 at 17:02
What if you take $Z$:= the closed linear span of some subset of the basis of $Y$ and {$x_0$}? –  Pietro Majer Apr 26 '12 at 17:44
@Pietro I want $Z\subseteq Y$. –  Mathy Apr 26 '12 at 17:49

If the values $\alpha_n$ are distinct and nonzero and none of the $\beta_n$ is zero, then there is no nontrivial invariant subspace of $Y$. To see this, identify $Y$ with $l^2$, so that $S$ takes the sequence $a = (a_n)$ to $\langle a, \beta\rangle \oplus (\alpha_n a_n) \in {\bf C} \oplus l^2$. Now $Sa \in Y$ if and only if $\langle a,\beta\rangle = 0$, in which case $Sa = 0 \oplus (\alpha_n a_n)$. So we seek a subspace $Z$ of $l^2$ consisting of vectors orthogonal to $\beta$, and which is invariant under multiplication by $\alpha$.

But if $Z$ is invariant under multiplication by $\alpha$, it is also invariant under multiplication by any power of $\alpha$, hence any polynomial in $\alpha$, hence any weak* limit of polynomials in $\alpha$. Assuming the $\alpha_n$ are distinct and nonzero, this means that $Z$ is invariant for all diagonal operators. So $Z$ has to be of the form $l^2(X) \subseteq l^2$ for some set $X \subseteq {\bf N}$. But then if none of the $\beta_n$ are zero, the only way for every vector in $Z$ to be orthogonal to $\beta$ is for $X$ to be empty. So $Z$ has to be the zero subspace.

-
The condition "$a_{n}$ nonzero" is not needed. –  jjcale May 11 '12 at 18:30
You mean "$\alpha_n$ nonzero". Right. –  Nik Weaver May 11 '12 at 21:39

Consider first a couple of special cases, where there is an obvious choice of $Z:=\operatorname{span}(z)$ with an eigenvctor $z\in Y$.

I. At least one of the $\beta_n$'s vanishes, say $\beta_{n_0}=0$. Then $z:=y_{n_0}$ is an eigenvector of $S$ on $Y$ .

II. At least two of the $\alpha_n$'s coincide, say $\alpha_{n_0}=\alpha_ {n_1}=\lambda$, for $n_0\neq n_1$. Then some non-trivial linear combination $p\beta_{n_0}+q\beta_{n_1}$ vanishes; $z:= p y_{n_0}+qy_{n_1}$ is an eigenvector with eigenvalue $\lambda$.

Conversely, if there is such an invariant subspace $Z\subset Y$, then the compact operator $S_{|Z}$ has an eigenvalue $\lambda$ with eigenvector $z:=\sum_{n=1}^\infty c_ny_n\in Y$. This means $$\sum_{n=1}^\infty \alpha_n c_ny_n + \left(\sum_{n=1}^\infty \beta_nc_n\right)x_0=\lambda \sum_{n=1}^\infty c_ny_n\\ ,$$ so $(\alpha_n-\lambda)c_n=0$ for all $n$ and $\sum_{n=1}^\infty \beta_nc_n=0$. Hence, if only one $c_n$ is non-zero, we are in case I; if there are at least two non-zero $c_n$, we are in case II.

-
@Pietro If $Z\subseteq Y$ is invariant, why does $S_{|Z}$ have an eigenvalue? It is possible that it is quasinilpotent and $0$ is not an eigenvalue. –  Mathy Apr 27 '12 at 13:27
Oops...there was something fishy indeed. I think I was following a king of circular argument. Deleting! –  Pietro Majer Apr 27 '12 at 17:20