Using the module of Kähler differentials, it is easy to show that $R\to S$ is formally unramified if and only if the induced maps $R\to S_{\mathfrak{p}}$ are formally unramified for all primes $\mathfrak{p}\subset S$.

Consider a presentation of $S$ over $R$ as $R[X]/I$ in generators and relations, where $R[X]:=R[X_m]_{m\in M}$ is a polynomial ring in a possibly infinite family of indeterminates indexed by $M$, and $I\subset R[X]$ is an ideal. Fix a family of generators of $I=(F_j)_{j\in J}$ indexed by $J$, again not necessarily finite.

It is enough to show that $R\to S$ is formally smooth. This is equivalent to showing that there exists a morphism of $R$-algebras that is a splitting for the canonical projection $\pi:R[X]/I^2 \to R[X]/I=S$, which will necessarily be unique because $R\to S$ is formally unramified.

Let $\bar{X}_m$ \overline{X}_m$ denote the image of $X_m$ in $R[X]/I^2$. We must find elements $\delta_m\in I/I^2$ such that $(\forall j\in J)F_j(X_m + \delta_m)$. delta_m)=0$. We rewrite this using Taylor's formula as $$\bar{F}_j+ \sum_{m\in M}\bar{\frac{\partial M}\overline{\frac{\partial F_j}{\partial X_m}}\delta_m=0.$$

Note: The bar should go over the entire partial factor but not $\delta_m$. This is a LaTeX error.

Rearranging, we get a system of equations indexed by $J$$$(*)_{j\in J}\sum_{m\in M}\bar{\frac{\partial M}\overline{\frac{\partial F_j}{\partial X_m}}\delta_m=-\bar{F}_j.$$X_m}}\delta_m=-\overline{F}_j.$$

We wish to find a unique solution for this system in the $\delta_m$. Since $\Omega_{S/R}=0$, each $dX_m\in \Omega_{R[X]/R}$ is an $S$-linear combination $dX_m=s_{m,1}dF_{j_{m,1}}+\cdots + s_{m,h_m}dF_{j_{m,h_m}}$. If we use the $s_{m,k}$ as coefficients to form $S$-linear combinations of the equations $(*)_{j_k}$, for each $m$, we get an equation of the form $$(**)_m \quad \delta_m=-(s_{m,1}\bar{F}_{j_{m,1}}+\cdots delta_m=-(s_{m,1}\overline{F}_{j_{m,1}}+\cdots + s_{m,h_m}\bar{F}_{j_{m,h_m}}).$$s_{m,h_m}\overline{F}_{j_{m,h_m}}).$$

Showing that these define solutions for all of the equations $(*)_j$ is not immediate, but it is a local question on $S$. However, our local rings $S_{\mathfrak{p}}$ are all formally étale, so the local conditions are satisfied. Then this proves the global claim.

(Note: This is not my proof. I've paraphrased the proof communicated to me by Mel Hochster.)

Edit: Fixed LaTeX using Scott's suggestion.

Using the module of Kähler differentials, it is easy to show that $R\to S$ is formally unramified if and only if the induced maps $R\to S_{\mathfrak{p}}$ are formally unramified for all primes $\mathfrak{p}\subset S$.

Consider a presentation of $S$ over $R$ as $R[X]/I$ in generators and relations, where $R[X]:=R[X_m]_{m\in M}$ is a polynomial ring in a possibly infinite family of indeterminates indexed by $M$, and $I\subset R[X]$ is an ideal. Fix a family of generators of $I=(F_j)_{j\in J}$ indexed by $J$, again not necessarily finite.

It is enough to show that $R\to S$ is formally smooth. This is equivalent to showing that there exists a morphism of $R$-algebras that is a splitting for the canonical projection $\pi:R[X]/I^2 \to R[X]/I=S$, which will necessarily be unique because $R\to S$ is formally unramified.

Let $\bar{X}_m$ denote the image of $X_m$ in $R[X]/I^2$. We must find elements $\delta_m\in I/I^2$ such that $(\forall j\in J)F_j(X_m + \delta_m)$. We rewrite this using Taylor's formula as $$\bar{F}_j+ \sum_{m\in M}\bar{\frac{\partial F_j}{\partial X_m}}\delta_m=0.$$

Note: The bar should go over the entire partial factor but not $\delta_m$. This is a LaTeX error.

Rearranging, we get a system of equations indexed by $J$$$(*)_{j\in J}\sum_{m\in M}\bar{\frac{\partial F_j}{\partial X_m}}\delta_m=-\bar{F}_j.$$

We wish to find a unique solution for this system in the $\delta_m$. Since $\Omega_{S/R}=0$, each $dX_m\in \Omega_{R[X]/R}$ is an $S$-linear combination $dX_m=s_{m,1}dF_{j_{m,1}}+\cdots + s_{m,h_m}dF_{j_{m,h_m}}$. If we use the $s_{m,k}$ as coefficients to form $S$-linear combinations of the equations $(*)_{j_k}$, for each $m$, we get an equation of the form $$(**)_m \quad \delta_m=-(s_{m,1}\bar{F}_{j_{m,1}}+\cdots + s_{m,h_m}\bar{F}_{j_{m,h_m}}).$$

Showing that these define solutions for all of the equations $(*)_j$ is not immediate, but it is a local question on $S$. However, our local rings $S_{\mathfrak{p}}$ are all formally étale, so the local conditions are satisfied. Then this proves the global claim.

(Note: This is not my proof. I've paraphrased the proof communicated to me by Mel Hochster.)