I just used the following.

Lemma. Let $A$ be a $\mathbb{Z}_p$-flat ring, of finite type over $\mathbb{Z}_p$, and suppose that $A \otimes \mathbb{F}_p$ is a domain. Then $A$ is a domain.

Proof: Suppose $ab = 0$ in $A$. Then one of $a, b$ must lie in $pA$, so we can write (without loss of generality) $a = p a_1$. Then by flatness $a_1 b=0$.

Continuing in this manner, we find that one of $a$ and $b$ must be infinitely divisible by $p$. But the finite type hypothesis implies that this is impossible unless one of $a$ or $b$ is in fact zero.

Given the statement, it seems like there should be a more conceptual reason why this should be true. Can anyone supply one? (A proof using more general facts in EGA counts as conceptual).

Edit: Kevin Buzzard gives a compelling reason why I have never seen this "fact" used before. Thank you both for your answers.

Edit 2: I suppose that replacing "finite type" with "p in the radical" would work (with an application of the Krull intersection theorem). In particular, the result is true as stated for a local $\mathbb{Z}_p$-algebra.

properflat scheme over a dvr, since the only open set which contains the special fiber is the entire space (in contrast with the disconnected counterexample suggested by Kevin). Ditto for the property of reducedness. – BCnrd May 4 '10 at 23:04