Let $f:X \to Y$ be a projective morphism between irreducible Noetherian schemes. If a fiber over a closed point of $Y$ is reduced is the generic fiber reduced?

2$\begingroup$ You may be interested to know that if $f:X \rightarrow Y$ is a proper flat map between noetherian schemes then the locus of $y \in Y$ such that $X_y$ is geometrically reduced is always open. This is EGA IV$_3$, 12.2.4 (which provides a wealth of openness results for properties of geometric fibers). In practice, it is properties of geometric fibers (rather than actual fibers) that define reasonable loci in the base (e.g., geometric irreducibility works well, whereas mere irreducibility isn't even a constructible condition on the base in general). $\endgroup$– MarguaxCommented Nov 10, 2013 at 22:20
2 Answers
Let's impose a flatness hypothesis (whose necessity is explained in Jason Starr's answer). The answer is still negative, but to explain the context for the counterexample it is instructive to first record some necessary features of any counterexample, so we know where to look.
In view of my above comment about geometric fibers, any search for a counterexample will necessarily have to involve a special fiber that is reduced yet not geometrically reduced. In other words, any counterexample will have to involve an imperfect residue field at the special point in the base.
Moreover, the answer is affirmative if the base $Y$ is Dedekind. Indeed, we can assume in such a case that the base $Y$ is Spec($R$) for a dvr $R$ with uniformizer $t$, so $X$ is a proper flat $R$scheme having reduced special fiber $X_0$. If $N$ is the coherent sheaf of nilpotents inside $O_X$ then for any section $f \in N(U)$ over an open $U \subset X$ we know that $f_0 = f_{U_0}$ vanishes since $X_0$ is reduced, so $f$ is a section of $tO_X$. In other words, $N \subset tO_X$. By $R$flatness of $X$, it is clear (check!) that $N \cap tO_X = tN$, so $N \subset tN$. Hence, by Nakayama's Lemma along points of the special fiber, it follows that $N$ has vanishing stalks along the special fiber $X_0 \subset X$. The closed support of the coherent $N$ inside the $R$proper $X$ is therefore disjoint from the special fiber. But every nonempty closed set in $X$ must meet $X_0$ due to $R$properness of $X$ and locality of $R$. Thus, the support is empty, which is to say $N = 0$. Then $X$ is reduced, so its localization given by the generic fiber over the integral base $Y$ is also reduced. That settles the question affirmatively when $Y$ is Dedekind (and $X$ is any proper flat $Y$scheme).
In view of the preceding observations, an "optimal" counterexample should involve a local base $Y = {\rm{Spec}}(R)$ where $R$ is a nonDedekind 1dimensional local noetherian domain having imperfect residue field with characteristic $p > 0$. And in fact there are counterexamples over such $R$, as we now construct. This shows that the EGA result with reducedness for geometric fibers is essentially "best possible". We will give counterexamples in equicharacteristic $p$. Maybe someone else can address the story with generic characteristic 0. (SEE THE END FOR SUCH AN EXAMPLE)
The idea is to make such an $R$ that is an order in a dvr of characteristic $p$ whose fraction field contains a certain $p$th root but for which $R$ does not contain that $p$th root. Let $k$ be a field of characteristic $p > 0$ and let $A = k(t)[x]_{(x)}$, a dvr with uniformizer $x$ and residue field $k(t)$ that is obviously not perfect. Let $F = {\rm{Frac}}(A) = k(t,x)$ and let $A' = A[T]/(T^p  t) = k(T)[x]_{(x)}$, so the dvr $A'$ with uniformizer $x$ is the integral closure of $A$ in $F' = F[T]/(T^p  t) = F(t^{1/p})$. Note that $A \rightarrow A'$ is an example of a finite extension of discrete valuation ring whose ramification degree is 1 but residual extension is not separable (so it is not "unramified").
The residue field of $A'$ is $k(T)$ in which the element $t$ from the residue field of $A$ has image $T^p$ that is a $p$th power, but we can "fix" that (or rather, "ruin" it) by considering the order $R = A + xA'$. Concretely, $R$ is the preimage of $k(t)$ under the reduction map $A' \twoheadrightarrow k(T)$.
Clearly $R$ is a 1dimensional local noetherian domain whose fraction field is $F'$, residue field is $k(t)$, and normalization is $A'$. Note that the element $T = t^{1/p}$ in the fraction field $F'$ of $R$ does not lie in $R$.
Inside $\mathbf{P}^2_A$ with homogeneous coordinates $[U,V,W]$, consider the closed subscheme $Z$ defined by $U^p + t V^p$. This is $A$flat since $A$ is Dedekind. Its special and generic fibers are the "same" projective scheme over the residue field $k(t)$ and fraction field $k(t,x)$ respectively, and as such these fibers are both reduced (as $t$ is not a $p$th power in either the residue field or fraction field of $A$, each of characteristic $p$) and are also both geometrically irreducible. The base change $X = Z_R$ is certainly $R$flat (since $Z$ is $A$flat) and projective with special fiber that is the same as that of $Z$ since $R$ and $A$ have the same residue field by design. In particular, $X_0$ is reduced (though not geometrically so!). The generic fiber $X_{F'} = Z_{F'}$ is nonreduced since $t$ is a $p$th power in $F'$. Note however that the $R$flat $X$ has irreducible generic fiber, so $X$ is also irreducible. Voila.
EDIT: Here is an even better example in the same spirit, but with a twist on the idea so that it works in generic characteristic 0. It will use an order in $\mathbf{Z}[T]_{(p)}$ as I had been hoping. This example was pointed out to me by somebody "offline".
Let $A = \mathbf{Z}[t]_{(p)}$, a dvr with uniformizer $p$, residue field $\mathbf{F}_p(t)$, and fraction field $F = \mathbf{Q}(t)$. Let $A'= \mathbf{Z}[T]_{(p)}$ made into an $A$algebra via $t \mapsto T^p$, so its fraction field $F'$ is $F(t^{1/p})$. Let $R = A + pA'$, a 1dimensional local noetherian domain with residue field $\mathbf{F}_p(t)$ and fraction field $F'$.
Consider the polynomial $f(U,V) = (U + T V)^p \in U^p + t V^p + pA'[U,V] \subset R[U,V]$. Obviously $f$ is a $p$th power over $F' = \mathbf{Q}(T)$, but it is not a (unit multiple of a) $p$th power over $R$ since the reduction of $f$ over the residue field of $R$ is $U^p + t V^p \in \mathbf{F}_p(t)[U,V]$, which is even irreducible.
Let $X \subset \mathbf{P}^1_R$ be defined by the vanishing of the homogeneous $f$. (The equicharacteristic$p$ example above ought to have been considered inside $\mathbf{P}^1$ just as well; I mistakenly thought being in $\mathbf{P}^2$ would provide better irreducibility properties, but those hold already with $\mathbf{P}^1$.) The $R$scheme is $R$flat because each of $U^p$ and $V^p$ have coefficients that are units in $R$ (ensuring that after dehomogenization, the coordinate ring of the affine open chart of $X$ is a finite free $R$module). In fact, we see that $X$ is finite flat over $R$, with degree $p$.
The special fiber of $X$ is the zero scheme of $U^p + t V^p$ in $\mathbf{P}^1$ over $\mathbf{F}_p(t)$, and this is visibly reduced. On the other hand, the generic fiber is $(U+TV)^p=0$ in $\mathbf{P}^1_{F'}$, and this is visibly not reduced (but is irreducible, and likewise $X$ is irreducible; in fact, $X_{\rm{red}} = {\rm{Spec}}(R)$!).

2$\begingroup$ this is a very nice answer ! $\endgroup$ Commented Nov 11, 2013 at 9:25

$\begingroup$ Thanks, Olivier. I hope someone can figure out if there is a counterexample with generic characteristic 0, such as over an order in $\mathbf{Z}[t]_{(p)}$ or something like that; it would be even nicer. :) $\endgroup$– MarguaxCommented Nov 11, 2013 at 15:58

1$\begingroup$ OK, thanks to a suggestions from someone offline, an example has now been given with generic characteristic zero. I have put it at the bottom of the answer. $\endgroup$– MarguaxCommented Nov 12, 2013 at 7:56

1

$\begingroup$ It is a pleasure to read your answers. $\endgroup$ Commented Nov 14, 2013 at 6:26
That is not true if you do not add some sort of flatness hypothesis. For a quick counterexample, let $Y$ be $\mathbb{A}^2$ with coordinates $(s,t)$. Let $\mathbb{P}^2$ have homogeneous coordinates $[U,V,W]$. Let $X$ be the closed subscheme of $\mathbb{A}^2\times \mathbb{P}^2$ with defining ideal $\langle W(tUsV), (tUsV)^2 \rangle$. Settheoretically, $X$ equals the Cartier divisor $Z(tUsV)$, which is irreducible. The generic fiber is nonreduced: it is the line in $\mathbb{P}^2$ with equation $tUsV=0$, but with an embedded point at the point $Z(tUsV,W)$. However, the schemetheoretic fiber over the origin in $\mathbb{A}^2$ is reduced: it is all of $\mathbb{P}^2$.
If you add a flatness hypothesis, this should follow from Auslander's Theorem, cf. EGA IV.6.11.

$\begingroup$ @Starr: The "problem" with applying Auslander's theorem is that it does not address fibral aspects, only properties of local rings on the "total space". $\endgroup$– MarguaxCommented Nov 11, 2013 at 0:16

$\begingroup$ @Marguax: I was assuming that the OP was asking about geometric fibers. In this case, I think Auslander's Theorem does apply. I can base change to a DVR whose generic point maps to the generic point of $Y$ and whose special point maps to the given closed point. So assume that $Y$ is regular. Then, if the closed fiber is reduced, or more generally $S_d$, then also the domain is $S_d$. Now apply Auslander's theorem. I would need to doublecheck in EGA, but I have a feeling this is roughly how the openness result is proved in EGA as well. $\endgroup$ Commented Nov 12, 2013 at 18:19

$\begingroup$ @Starr: Ah, I didn't realize you had in mind just geometric fibers. Your proposed argument (and sense of how it probably goes in EGA) sounds reasonable to me. $\endgroup$– MarguaxCommented Nov 12, 2013 at 19:58