A diffeomorphism of complex surfaces mapping subvarieties to subvarieties

Question

Let $X$ and $Y$ be smooth projective complex surfaces. If a diffeomorphism from $X$ to $Y$ maps subvarieties to subvarieties does it have to be holomorphic or antiholomorphic? Can we at least verify this for K3 surfaces?

Answer 1

I am posting this answer because the following linear algebra proposition is too long for a comment.

Lemma. Let $A$, respectively $B$, be an invertible $\mathbb{R}$-linear operator on $\mathbb{C}^2$ that is $\mathbb{C}$-linear, resp. $\mathbb{C}$-conjugate linear.
(1). For every $1$-dimensional $\mathbb{C}$-linear subspace $V\subset \mathbb{C}^2$ such that $A(V)$ equals $B(V)$, there exists a generator $\mathbf{v}\in V$ such that $B(\mathbf{v})=r\cdot A(\mathbf{v})$ for a positive real $r$.
(2). For every ordered pair $(\mathbf{v},\mathbf{w})$ of $\mathbb{C}$-linearly independent elements of $V$ with $B(\mathbf{v})=r\cdot A(\mathbf{v})$ and $B(\mathbf{w})=s\cdot A(\mathbf{w})$ for positive reals $r,s$, for every ordered pair $(\lambda,\mu)\in \mathbb{C}^2$, the elements $A(\lambda\cdot \mathbf{v} +\mu\cdot \mathbf{w})$ and $B(\lambda\cdot \mathbf{v} +\mu\cdot \mathbf{w})$ are $\mathbb{C}$-linearly independent if and only if $r\overline{\lambda}\mu$ equals $s\lambda \overline{\mu}$.

Proof. (1). For any generator $\mathbf{u}\in V$, there exists $\rho \in \mathbb{C}^\times$ such that $B(\mathbf{u})$ equals $\rho\cdot A(\mathbf{u})$. For every complex unit $\zeta$, also $B(\overline{\zeta}\cdot \mathbf{u})$ equals $\zeta^2 \rho A(\overline{\zeta}\cdot \mathbf{u})$. For an appropriate choice of $\zeta$, the complex number $\zeta^2 \rho$ is a positive real number.

(2). This is a straightforward computation. QED

Proposition. Let $A$, respectively $B$, be a nonzero $\mathbb{R}$-linear operator on $\mathbb{C}^2$ that is $\mathbb{C}$-linear, resp. $\mathbb{C}$-conjugate linear. If $A(\mathbf{u})$ and $B(\mathbf{u})$ are $\mathbb{C}$-linearly dependent for every $\mathbf{u}\in \mathbb{C}^2$, then $A(\mathbb{C}^2)$ and $B(\mathbb{C}^2)$ are equal and have complex dimension $1$.

Proof. By the lemma, if $A$ and $B$ are both invertible, and if $V,W$ are distinct $1$-dimensional $\mathbb{C}$-linear subspaces of $\mathbb{C}^2$ with $A(V)=B(V)$ and $A(W)=B(W)$, then there exist generators $\mathbf{v}\in V$ and $\mathbf{w}\in W$ and positive reals $r,s$ such that $A(\lambda\cdot \mathbf{v} + \mu\cdot \mathbf{w})$ and $B(\lambda\cdot \mathbf{v} + \mu\cdot \mathbf{w})$ are $\mathbb{C}$-linearly dependent if and only if $r\overline{\lambda}\mu$ equals $s\lambda \overline{\mu}$. In particular, for $\lambda = 1$ and $\mu=i$, this does not hold, thus the vectors are not $\mathbb{C}$-linearly dependent (they are $\mathbb{C}$-linearly independent). Therefore, for $A$, $B$ as in the statement, at least one of $A$ and $B$ is not invertible.

Thus, there exists a nonzero vector $v\in \mathbb{C}^2$ such that $A(v)=0$ or $B(v)=0$, i.e., either $A(\mathbb{C}^2)$ or $B(\mathbb{C}^2)$ is a $1$-dimensional complex linear subspace of $\mathbb{C}^2$. By the hypothesis on $A$ and $B$, then both $A(\mathbb{C}^2)$ and $B(\mathbb{C}^2)$ are equal to this $1$-dimensional complex linear subspace. QED

Corollary. Let $A$, respectively $B$, be a nonzero $\mathbb{R}$-linear transformation from $\mathbb{C}^n$ to $\mathbb{C}^m$ that is $\mathbb{C}$-linear, resp. $\mathbb{C}$-conjugate linear. If the $\mathbb{R}$-linear transformation $D=A+B$ sends $1$-dimensional $\mathbb{C}$-linear subspaces of $\mathbb{C}^n$ to $\mathbb{C}$-linear subspaces of $\mathbb{C}^m$, then $A(\mathbb{C}^n)$ and $B(\mathbb{C}^n)$ are equal and have $\mathbb{C}$-dimension equal to $1$.

Proof. For the restrictions of $A$ and $B$ to a general $2$-dimensional $\mathbb{C}$-linear subspace of $\mathbb{C}^n$ (considered as an $\mathbb{R}$-linear transformation to the image $2$-dimensional $\mathbb{C}$-linear subspace of $\mathbb{C}^m$), the hypotheses of the proposition hold. In particular, the restrictions of $A$ and $B$ have rank $1$ with equal image. Therefore $A(\mathbb{C}^n)$ and $B(\mathbb{C}^n)$ are equal and have $\mathbb{C}$-dimension equal to $1$. QED

Theorem. Let $X$ be the complex manifold underlying a proper, smooth, connected $\mathbb{C}$-scheme. Let $Y$ be a complex analytic space. Let $f:X\to Y$ be a differentiable function whose $\mathbb{R}$-rank is $>2$ on a connected, dense open subset of $X$. If $f$ maps complex algebraic subvarieties of $X$ to complex analytic subvarieties of $Y$, then either $f$ is holomorphic or anti-holomorphic.

Proof. For every point $p$ in the connected, dense open subset, the derivative map $D=d_p f$ from $T_p X$ to $T_{f(p)}Y$ has rank $>2$. Of course $D$ equals $A+B$ for $A=(D - J_{Y,f(p)}\circ D \circ J_{X,p})/2$ and $B=(D+J_{Y,f(p)}\circ D\circ J_{X,p})/2$, and $A$, respectively $B$, is $\mathbb{C}$-linear, resp. $\mathbb{C}$-conjugate linear.

By the hypothesis on $X$, every $1$-dimensional $\mathbb{C}$-linear subspace of $T_p X$ equals the tangent space to a closed algebraic curve $Z\subset X$ that is smooth at $p$. By the hypothesis, $f(Z)$ is complex analytic, so that $d_p f$ maps this $1$-dimensional $\mathbb{C}$-linear subspace of $T_pX$ to a $\mathbb{C}$-linear subspace of $T_{f(p)} Y$. By the corollary, either $d_p f$ is $\mathbb{C}$-linear or $\mathbb{C}$-conjugate linear, i.e., either $A$ or $B$ is zero.

Since both $A$ and $B$ vary continuously in $p$, either $A$ equals $0$ for all $p$ in the dense open subset of $X$, or $B$ equals $0$ for all $p$ in the dense open subset of $X$. Since the open subset is dense, again by continuity, either $d_p f$ is $\mathbb{C}$-linear for all $p\in X$ or $d_pf$ is $\mathbb{C}$-conjugate linear for all $p\in X$. In other words, either $f$ is holomorphic or anti-holomorphic. QED

Remark. Obviously this is false without the hypothesis that the $\mathbb{R}$-rank of $f$ is $>2$. Indeed, post-compose any nonconstant holomorphic function from $X$ to a Riemann surface $Y$ with a general diffeomorphism of $Y$ to obtain a counterexample.