This is an example when proving "locally free" instead of merely "flat" is easier and more straightforward, and no Noetherian assumption on the base is needed. The point is that if some coefficient $a$ of a polynomial $f\in R[x_1,\dotsc x_n]$ is nonzero at $p\in Spec R$ (i.e. nonzero in $R/p$) then it is invertible in an open neighborhood $D(a)\ni p$.
So let $f\in R[x_1,\dotsc,x_n]$ be a polynomial of degree $d$, $p$ be a point of $Spec R$ (i.e. a prime ideal in $R$) and $k$ be the quotient field of $R/p$. Let $\bar f \in k[x_1,\dotsc,x_n]$ be the reduction of $f$ modulo $p$.
Using a change of coordinates in $k[x_1,\dotsc,x_n]$, put $\bar f$ in a Weierstrass form w.r.t. to the variable $x_n$. This means that
$$ \bar f= \bar a x_n^d + p_{d-1}x_n^{d-1} + \dots + p_0 $$
for some polynomials $p_j$ in the remaining variables $x_1,\dotsc,x_{n-1}$, and $\bar a\in k$, $\bar a\ne 0$.
If $k$ is infinite, this can be done by a linear change of coordinates. If $k$ is finite, there is a little trick.
If $r_i/s_i\in k$ are the coefficients involved in the change of coordinates ($r_i,s_i\in R$) then this change of coordinates can be done already in the ring $R'=R[1/a \prod s_i]$, i.e. over the open set $Spec R'= D(a\prod s_i)$ in $Spec R$ containing $[p]$. Further, $a$ is invertible over this set.
Now, over $R'$ the quotient $R'[x_1,\dotsc,x_n]/(f)$ is a free $R'[x_1,\dotsc,x_{n-1}]$-module with a basis $1,x_n,\dotsc, x_n^{d-1}$. Hence, it is a free $R'$-module. QED
This proves the statement for a family of nonzero hypersurfaces in $\mathbb A^n$. For a family of nonzero hypersurfaces in $\mathbb P^n$, cover $\mathbb P^n$ by $\mathbb A^n$ appropriately.