I am just posting the comment as an answer. One reference is EGA $\textrm{IV}_2$, Section 6.11, pp. 158-163.
Edit. Hans points out that in EGA only the absolute version of the results are proved, whereas he is asking about the relative version. However, the same proofs as in that section prove the relative result as well, almost verbatim. Here is a link to an extract of a preprint of Chenyang Xu and I where the details are made explicit, http://www.math.stonybrook.edu/~jstarr/papers/RSCff_02_23_13a.pdf
Added later. I now see that EGA also addresses the relative case in EGA $\textrm{IV}_3$, Section 12.1, pp. 174-179, particularly Théorème 12.1.1.
Second Edit. I realized that, since Hans wants the result locally on the target (i.e., base of the family of closed subschemes) rather than locally on the domain (which is what is proved in EGA), there is a much simpler proof. Let $R$ be a Noetherian ring. Let $Z\subset \mathbb{P}^n_R$ be a closed subscheme such that $p_Z:Z\to \text{Spec}(R)$ is flat. By the Noether Normalization Theorem, after passing to an affine open cover of $\text{Spec}(R)$, there exists a linear projection of $R$-schemes, $f:Z\to \mathbb{P}^d_R$, that is finite and surjective. Then $Z$ is Cohen-Macaulay over $R$ if and only if $f$ is flat. By openness of flatness, there is an open subscheme $U$ of $\mathbb{P}^d_R$ over which the coherent sheaf $f_*\mathcal{O}_Z$ is locally free. Let $C$ be the closed complement of $U$. The image of $C$ in $\text{Spec}(R)$ is a closed subset of $\text{Spec}(R)$. The relevant open subset of $\text{Spec}(R)$ is the open complement of this closed subset.
Hans asks in the comments about openness of flatness for the morphism $f$. This can be proved directly. In EGA $\textrm{IV}_3$, Section 12.3, pp. 183-187 this is proved. Take $A$ to be the base ring $R$, take $B$ to be the ring of regular functions on an open affine $U$ in $\mathbb{P}^n_R$, and take $M$ to be the regular sections of $f_*\mathcal{O}_Z$ over $U$.