I am interested in a morphism of $S$-schemes $f : X \to Y$ such that $X$ and $Y$ are proper over $S$ and there is some $s_0 \in S$ such that $f : X_{s_0} \to Y_{s_0}$ is a closed immersion. Is it true that there is an open neighborhood $U \subset S$ of $s_0$ so that $X_U \to Y_U$ is a closed immersion?

Because $f$ is automatically proper, I hoped to use the characterization of closed immersions as proper monomorphisms. Using an argument on formal neighborhoods of the fibers, I think I can show that $f : X \to Y$ is unramified at each point of the fiber and hence should be unramified on some neighborhood of the fiber. Maybe I can also show that $f$ is radicial in a similar fashion?

I am okay with assuming that $X, Y$ are noetherian and flat over $S$ if this helps.