(1) A family of hypersurfaces of degree $d$ over an affine scheme $Spec\ R$ means the following: it is a closed subscheme of $\mathbb P^n_R$ given by a homogeneous polynomial $f(x_0,\dots,x_n)$ of degree $d$ satisfying the following condition:

> *Every fiber is a hypersurface of degree $d$.* This means that for every prime ideal $m\subset R$, the reduction $\bar f\in k[x_0,\dots,x_n]$ is a polynomial of degree $d$, where $k=R/m$.

Now, let $m$ be a maximal ideal, so that $k$ is a field, and look at the graded ring $k[x_0,\dots,x_n]/(f)$. For each $k\ge d$, the degree-$k$ part is a vector space of dimension $\binom{k+n}{n}-\binom{k-d+n}{n}$. Thus, some $\binom{k-d+n}{n}$ monomials can be written as linear combinations, with coefficients in $k$, of the remaining monomials. This is done by solving a system of linear equations obtained by setting $x^m f=0$, where $x^m$ are monomials of degree $k-d$.

Now, consider the same system of linear equations with coefficients in $R$. For each of the $\binom{k-d+n}{n}$ monomials as above you get a principal minor $M$ of your matrix, and the reduction of $\det M$ in $k$ is not zero. Thus, over an open set $Spec\ R[1/\det M]$, this determinant is invertible, and the monomial can be eliminated.

Thus, over an open subset $Spec\ A$ of the point $[m]\in Spec\ R$, the degree-$k$ part of the ring $S[x_0,\dots,x_n]/(f)$ is a **free** $A$-module. Recalling how $Proj$ is covered by $Spec$'s and that a free module is flat, this implies that $Proj\ S[x_0,\dots,x_n]/(f)$ is flat over $Spec\ S$. (We will assume $R$ and so $A$ to be Noetherian here for simplicity.)

This is the main trick for proving flatness over a non-reduced base: you prove freeness instead. For a finitely generated module over a Noetherian ring, flatness and freeness are equivalent. So for a projective morphism $f:X\to Y$ a coherent sheaf $F$ on $X$ is flat over $Y$ iff the sheaves $f_* F(k)$ on $X$ are locally free for $k\gg0$.

In (2), you are mistaken about Hartshorne: Theorem III.12.11 (Cohomology and Base Change) has no assumption for the base to be reduced. So if $H^i(X_y,F_y)=0$ for $i>0$ then $f_*F$ is locally free, for any (Noetherian) base $Y$ and coherent sheaf $F$ on $X$, flat over $Y$.