As noted by Jason Starr, the Generic Principle has been worked out in details by Grothendieck in EGA IV. If your French is not on top these days, the following version looks quite appealing (and can be found as Theorem 23.9 and 24.4 in Matsumura's Commutative Ring Theory) :
Let $f : X \longrightarrow Y$ be a flat (this is the expected fibration-like hypothesis), finite type morphism of Noetherian schemes with $X$ irreducible and let $y \in Y$.
$\bullet$ if $Y$ and $f^{-1}(y)$ satisfy the condition $R_k$, then there is a dense open subset $X' \subset X$ which satisfies the condition $R_k$.
$\bullet$ if $Y$ and $f^{-1}(y)$ satisfy the condition $S_k$, then there is a dense open subset $X' \subset X$ which satisfies the condition $S_k$.
Note that reduced is $R_0$ + $S_1$ (see for instane Lemma 10.157.3 in The Stack Project), so that your expectation in the Metaquestion is correct.
EDIT (added after the OP accepted the answer) : As the OP was hoping for a proof that could be understood by a beginning graduate student, I provide two other proofs, which are probably more basic that the one based on the results in Matsumura's book I quoted above.
$\boxed{1} \ $ If $k$ is assmued to be perfect, one can prove a more general statement. Namely : let $f : X \longrightarrow Y$ be a morphism between finite type $k$-schemes and let $y \in Y$ such that $Y$ is smooth at $y$. Assume that $\dim f^{-1}(y) = \dim X - \dim Y$ and that there exists $x \in f^{-1}(y)$ such that $f^{-1}(y)$ is smooth at $x$, then there exists an open neighborhood of $x \in X$ where $X$ is smooth.
Proof : Consider the cotangent exact sequence:
$$f^{*} \Omega_{Y/k} \rightarrow \Omega_{X/k} \rightarrow \Omega_{X/Y} \rightarrow 0 $$
As the relative cotangent sheaf satisfies base change, we have:
$$\Omega_{X/Y} \otimes k(x) = \Omega_{f^{-1}(y)/k} \otimes k(x).$$
Hence, tensoring the above exact sequence by $k(x)$, we get:
$$ f^{*} \left(\Omega_{Y/k} \right) \otimes k(x) \rightarrow \Omega_{X/k} \otimes k(x) \rightarrow \Omega_{f^{-1}(y)/k} \otimes k(x) \rightarrow 0. $$
Since the fiber $f^{-1}(y)$ is smooth at $x$ and of dimension $\dim X - \dim Y$, we have $\dim_k \Omega_{f^{-1}(y)/k} \otimes k(x) = \dim X - \dim Y$. We also know that $Y$ is smooth at $y$, so that $\dim_k f^{*} \left(\Omega_{Y/k} \right) \otimes k(x) = \dim Y$. The above exact sequence then implies $\dim_k \Omega_{X/k} \otimes k(x) \leq \dim X$. Hence, $\dim \Omega_{X/k} \otimes k(x) = \dim X$. The field $k$ being perfect, there exists a non-empty open neighborhood $x \in U \subset X$ scuh that $\Omega_{X/k}$ is locally free on $U$ and then $X$ is smooth along $U$.
$\boxed{2} \ $ With no assumption on $k$, we can prove the following : let $f : X \longrightarrow Y$ be a finite dominant morphism between finite type $k$-schemes. Assume that $Y$ is integral and that there is $y \in Y$ such that $f^{-1}(y)$ is scheme-theoretically equal to $q$ reduced points (say $x_1, \ldots, x_q$). Then, we can find an open neighborhood of say $x_1 \in U \subset X$ where $X$ is reduced.
Proof : The result being local on $X$ and $Y$, we can assume that both $X$ and $Y$ are affine. Let $A \longrightarrow B$ a finite monomorphism of finite-type $k$-algebras. We assume that $A$ is reduced and we let $\mathfrak{m}$ a maximal ideal of $A$ such that $B \otimes \mathfrak{m} = k^{q}$. Let $f_1, \ldots, f_q$ be a basis of $B/\mathfrak{m}$ as a $k$ vector space and $e_1, \ldots, e_q$ elements in $B$ whose images in $B/ \mathfrak{m}$ equal the $f_i$. We also denote by $e_i$ the images of the $e_i$ in the local ring $B_{\mathfrak{m}}$. The $e_i$ induces a morphism of $A_{\mathfrak{m}}$-modules of finite type:
$$ \Phi : A_{\mathfrak{m}}^q \longrightarrow B_{\mathfrak{m}}.$$
Since the morphism induced by $\Phi$ betwen the corresponding quotient rings (by $\mathfrak{m}$) is an isomorphism, the Nakayama's lemma allows to conclude that $\Phi$ is an epimorphism. We also know that $A$ is a domain so that the $A_{\mathfrak{m}}$-module of finite type $A_{\mathfrak{m}}^q$ is torsion-free. We then easily check that $\operatorname{Ker}(\Phi)$ is a finite type $A_{\mathfrak{m}}$-module which satisfies :
$$\mathfrak{m}.\operatorname{Ker}(\Phi) = \operatorname{Ker}(\Phi).$$
The Nakayama's lemma implies $\operatorname{Ker}(\Phi) = \{0\}$ and $\Phi$ is a $K$-algebra isomorphism between $A_{\mathfrak{m}}^q$ and $B_{\mathfrak{m}}$. In particular, $B_{\mathfrak{m}}$ is reduced.
Let us finally check that there exists an open affine of $\operatorname{Spec}(B)$ which is equally reduced. Let:
$$\Psi : B \longrightarrow B_{\mathfrak{m}},$$
be the loalization morphism. Its kernel, an ideal of $B$, is finitely generated, say by $h_1, \ldots, h_q$. By definition, the $h_i$ are $0$ in $B_{\mathfrak{m}}$ if and only if we can find $s_i \in B \backslash \mathfrak{m}$ such that $s_i.h_i = 0 \in B$.Let then $s =s_1 \times \ldots \times s_q$. The ideal $\mathfrak{m}$ being prime, we have $s \in B \backslash \mathfrak{m}$, wo that $\Psi$ induces a morphism of local ring:
$$\Psi_s : R_{s} \rightarrow R_{\mathfrak{m}}.$$
The kernel of $\Psi_s$ is the image of the kernel of $\Psi$ in $R_s$ : this equal to $\{0\}$ by definition of $s$. The map $\Psi_s$ is then a monomorphims, so that $R_s$ is a subring of a reduced ring : it is a reduced ring!
