Relative integral closure

Question

Definition: Let $R$ be an $A$-algebra, where $R$ and $A$ are both commutative rings with unit. Let $S \subset R$ be the (multiplicatively closed) subset consisting of those nonzerodivisors $s \in R$ such that $R/sR$ is flat over $A$. I will call $R$ integrally closed over $A$ if it satisfies

1. $R$ is flat over $A$, and
2. $R$ is integrally closed in $S^{-1}R$.

Is $A[x_1, \dotsc, x_n]$ necessarily integrally closed over $A$?

If it is helpful, you may assume $A$ is noetherian, or even of finite type over a field. But do not assume it is integral or reduced or anything like that. (In the case I care about, $A$ is likely to be an open affine on a Hilbert or Quot scheme.)

Some thoughts: assuming things are noetherian, and I have not made an error, the condition for $s \in S$ is equivalent to the condition that $s$ pulls back to a nonzerodivisor on every fiber of $\operatorname{Spec} R$ over $\operatorname{Spec} A$. This, in turn, is equivalent to the condition that $V(s)$ contains no associated component of any fiber.

If this "relatively integrally closed" condition is preserved under pullbacks, then the question has a positive answer. However, the only reason I might even imagine that this is true is my hope that the condition I have given is a reasonably "natural" condition for a morphism to have. I can't think of a way to begin arguing for preservation under pullbacks, and I don't even find it all that plausible.

Another, more plausible (to me) statement that would imply a positive answer is if normal morphisms (in the sense of Grothendieck) are necessarily "relatively integrally closed." A morphism is normal if it is flat and its (geometric) fibers are normal. I have a flawed argument for this: let $\ell \in S^{-1}R$ be integral over $R$. Then over each (geometric) fiber, $\ell$ is still integral over $R$, so $R \to R[\ell]$ pulls back to an isomorphism over each (geometric) fiber, and consequently must have been an isomorphism to begin with. The flaw (that I've found and cannot seem to repair) is that, at least in principle, $R[\ell] \to S^{-1}R$ need not pull back to an injection.

---

Note: Will Sawin's answer below was incredibly helpful to me. Please upvote it.

Answer 1

$S$ consists of polynomials whose coefficients are not in any ideal, that is, polynomials whose coefficients generate the unit ideal.

First assume $A$ is integral. Let $K$ be the field of fractions of $A$, then clearly $K[x_1,...,x_n]$ is an integrally closed unique factorization domain. Given an element in this that is in $S^{-1}R$, we seek to prove it is in $R$. We're writing a polynomial over $K$ as a fraction $x/y$ of polynomials over $A$. To make it a polynomial, every prime in $y$ must be paired with a prime in $x$. Their ratio is an element of $K$, such that multiplied by every coefficient of the prime in $x$, that is, every generator of the unit ideal, gets an element in $A$. Therefore it must be an element of $A$. So you can cancel them and leave an element of $A$ in $X$. This allows you to cancel everything in $x$, including constant terms, which must be units. We are left with a polynomial in $R$.

Now suppose $A$ is not integral. We're going to attempt to do this analysis separately on every prime ideal. Let $T$ be the integral closure of $R$ in $S^{-1}R$. The prime ideals of $A$ extend to $T$. Let $t$ be an element of $T$, then $A/p$ is an integral domain and so $t$ is an element of $R$ plus something in $p$. Given another prime, $q$, we "glue together" these characterizations. We have a residue in $R/p$ and a residue in $R/q$, which agree in $R/(p+q)\subset T/(p+q)$. This makes $t$ an element of $A$ mod any finite set of primes, by a stronger version of the Chinese Remainder Theorem. This is enough for reduced Noetherian schemes - choose the generic points of the irreducible components of $\textrm{Spec} A$.

EDIT: As Charles pointed out, this fails for any non-reduced scheme. Adjoin two invariants $x,y$, let $\epsilon$ be nilpotent, then $\epsilon x/y$ is nilpotent and so integral, but $\epsilon x$ is not a multiple of $y$ in $R$.