A smooth map on a Banach manifold whose pointwise rank is finite but its rank is not globally bounded

Question

Is there a connected Banach manifold $M$ and a smooth map $f:M \to M$ such that the rank of $Df_x$ is finite for every $x\in M$ but this rank is not uniformly bounded

Answer 1

Let $\mathbb H$ and $(e_n)_{n\geq0}$ be your favourite second countable Hilbert space and orthonormal Hilbert basis. Mine is the space $\ell^2(\mathbb N)$ of square integrable sequences, with $e_n$ the indicator function of the singleton $\lbrace n\rbrace$. I write, for any $x\in\mathbb H$, $x^n$ for the $n$th coordinate $\langle e_n,x\rangle$ of $x$.

Choose a smooth function $\phi:\mathbb R\to\mathbb R$ such that $\phi$ is identically zero over $(-\infty,1/2)$ and $\phi'(1)\neq0$. Then $$f:x\mapsto\sum_{n\geq0} \phi(x^n)e_n$$ is a function as you describe. Indeed,

• if $|x^n|<1/3$ for all $n\geq N$, then in a neighbourhood of $x$ the condition still holds with $1/2$ instead of $1/3$, and the function $f$ has values in the finite dimensional space generated by $(e_0,\cdots,e_{N-1})$ hence its differential has finite rank;

• since $x$ can have $|x^n|\geq1/3$ for only a finite number of $n$, $f$ is well-defined and smooth;

• the differential of $f$ at $x=e_0+\cdots+e_N$ is $$Df_x:u\mapsto\sum_{0\leq n\leq N} \phi'(1)u^ne_n,$$ which has rank $n$.

Just for fun, here is a stronger topological result. Say that $f$ has finite rank if $Df_x$ has finite rank for all $x$, and bounded rank if the rank of $Df_x$ is bounded uniformly in $x$. We say that $x$ is a nice point for $f$ if there exists a neighbourhood of $x$ over which $f$ has bounded rank, otherwise we say it is bad.

Theorem

Let $E$ be a subset of $\mathbb H$. Then there exists $f:\mathbb H\to\mathbb H$ of finite rank whose set of nice points is exactly $E$ if and only if $E$ is open dense.

This makes it possible to have many bad points, for instance there exists an $f$ with finite rank such that for all $\varepsilon>0$, there exists $x$ in the unit ball such that $B(x,\varepsilon)$ contains infinitely many bad points.

The direct implication is a consequence of the Baire category theorem. Indeed, the set $E_n$ of points $x$ such that $Df_x$ has rank at most $n$ is closed, and the set $E=\bigcup_{n\geq0}\operatorname{int}E_n$ of points where the rank is locally bounded is clearly open. To show that $E$ is dense, fix a non-empty open set $U$. Since $f$ has finite rank, $\bigcup_{n\geq0}(E_n\cap U)=U$, hence one of the $E_n\cap U$ must have non-empty interior, which is the same as saying that $\operatorname{int}E_n\cap U$ is non-empty.

In the other direction, this answer shows that in a Hilbert space, any closed set is the vanishing set of some smooth real-valued map with all derivatives uniformly bounded ($\forall k,\exists M,\forall x,\|D^k\rho_x\|\leq M$). Let $d:\mathbb H\to\mathbb R$ be the distance to $E^\complement$; note that it is continuous. Let $E_0$, resp. $E_n$ for $n>0$, be the inverse image by $d$ of $(1,+\infty)$, resp. $(\frac1{n+1},\frac2n)$. Obviously, $E_n$ is open for all $n$ and $E$ is the union of the $E_n$. Moreover, since $E$ is open dense, $x\in E$ if and only if there exists a neighbourhood of $x$ that intersects finitely many of the $\overline{E_n}$.

Let $\rho_n:\mathbb H\to\mathbb R$ be a smooth function with zero set $E_n$ and all derivatives uniformly bounded. Then $$ x\mapsto \rho_n(x)\sum_{k\leq i\leq \ell}x^ie_i $$ has rank zero in the complement of $\overline{E_n}$, and at least $k-\ell$ on $E_n$ (because the derivative of $\rho_n$ may only “kill” at most one direction in the image of the projector). Moreover, all its derivatives are uniformly bounded. It means that there exists a sequence $(\varepsilon_n)_{n\geq0}$ of positive numbers such that all the derivatives of the series $$ f:x\mapsto\sum_{n\geq0}\varepsilon_n\rho_n(x)\left(x^{n^2+1}e_{n^2+1}+\cdots+x^{(n+1)^2}e_{(n+1)^2}\right) $$ converge uniformly over $\mathbb H$, so that $f$ is well-defined and smooth (see the reference above for more details).

The rank of $Df_x$ is at least $2n$ but finite over $E_n$, and the rank of $Df_x$ is zero (hence finite) at all points $x$ of $E^\complement$. Moreover, $x$ is nice for $f$ if and only if one of its neighbourhoods intersects only finitely many $\overline{E_n}$, if and only if $x\in E$.