Welcome new contributor. This is too long for a comment. Let $k$ be a field. Let $R$ be $k[[t]]$. Let $C$ be the $R$-algebra $$C=R[x_n:n\in \mathbb{Z}_{\geq 0}].$$ Let $J$ be the ideal in $C$, $$J=\langle x_n-tx_{n+1} :n\in \mathbb{Z}_{\geq 0} \rangle.$$ Let $A$ be the $R$-algebra $C/J$.

Since $A$ has no $t$-torsion, the $R$-module $A$ is $R$-flat. The fiber ring $A\otimes_R (R/t R)$ equals $R/tR$, which is smooth over $R/tR$. The fiber ring $A\otimes_R R[1/t]$ equals $R[1/t][x_0]$, which is smooth over $R[1/t]$.

Now consider the $R$-algebra $B=C/J^2$. Let $I$ be the image ideal in $B$, $$I=J/J^2.$$ This is a square-zero ideal whose quotient algebra equals $A$. Thus, if $A$ is formally smooth, there exists a retraction of $R$-algebras, $$s:A\to B.$$ In particular, $s(x_0)$ is a $t$-divsible element of $B$ that maps to $x_0$ in the quotient algebra $A$.

Consider the quotient of $B$ by the ideal $$K = \langle x_mx_n :m,n\in \mathbb{Z}_{\geq 0} \rangle.$$ The quotient $B$-algebra equals $R\oplus M$ where $M$ is a square-zero ideal that is the free, countably-generated $R$-module with basis $$\{ x_n : n\in \mathbb{Z}_{\geq 0} \}.$$ In particular, the element $x_0$ in this quotient algebra is not $t$-divisible in $M$. Thus, there is no $t$-divisible element of $B$ that maps to $x_0$ in the quotient algebra $A$. Therefore, there is no retraction $s$, and $A$ is not formally smooth.

Welcome new contributor. This is too long for a comment. Let $R$ be a DVR with a uniformizing element $t$, e.g., $k[[t]]$ where $k$ is a field. Let $C$ be the $R$-algebra $$C=R[x_n:n\in \mathbb{Z}_{\geq 0}].$$ Let $J$ be the ideal in $C$, $$J=\langle x_n-tx_{n+1} :n\in \mathbb{Z}_{\geq 0} \rangle.$$ Let $A$ be the $R$-algebra $C/J$.

Since $A$ has no $t$-torsion, the $R$-module $A$ is $R$-flat. The fiber ring $A\otimes_R (R/t R)$ equals $R/tR$, which is smooth over $R/tR$. The fiber ring $A\otimes_R R[1/t]$ equals $R[1/t][x_0]$, which is smooth over $R[1/t]$.

Now consider the $R$-algebra $B=C/J^2$. Let $I$ be the image ideal in $B$, $$I=J/J^2.$$ This is a square-zero ideal whose quotient algebra equals $A$. Thus, if $A$ is formally smooth, there exists a retraction of $R$-algebras, $$s:A\to B.$$ In particular, $s(x_0)$ is a $t$-divsible element of $B$ that maps to $x_0$ in the quotient algebra $A$.

Consider the quotient of $B$ by the ideal $$K = \langle x_mx_n :m,n\in \mathbb{Z}_{\geq 0} \rangle.$$ The quotient $B$-algebra equals $R\oplus M$ where $M$ is a square-zero ideal that is the free, countably-generated $R$-module with basis $$\{ x_n : n\in \mathbb{Z}_{\geq 0} \}.$$ In particular, the element $x_0$ in this quotient algebra is not $t$-divisible in $M$. Thus, there is no $t$-divisible element of $B$ that maps to $x_0$ in the quotient algebra $A$. Therefore, $A$ is not formally smooth over $R$.

Welcome new contributor. This is too long for a comment. Let $k$ be a field of characteristic $2$. Let $R$ be $k[[t]]$. Let $C$ be the $R$-algebra $$C=R[x_n:n\in \mathbb{Z}_{\geq 0}].$$ Let $J$ be the ideal in $C$, $$J=\langle x_n-tx_{n+1} :n\in \mathbb{Z}_{\geq 0} \rangle.$$ Let $A$ be the $R$-algebra $C/J$.

Since $A$ has no $t$-torsion, the $R$-module $A$ is $R$-flat. The fiber ring $A\otimes_R (R/t R)$ equals $R/tR$, which is smooth over $R/tR$. The fiber ring $A\otimes_R R[1/t]$ equals $R[1/t][x_0]$, which is smooth over $R[1/t]$.

Now consider the $R$-algebra $B=C/J^2$. Let $I$ be the image ideal in $B$, $$I=J/J^2.$$ This is a square-zero ideal whose quotient algebra equals $A$. Thus, if $A$ is formally smooth, there exists a retraction of $R$-algebras, $$s:A\to B.$$

Consider the quotient of $B$ by the ideal $$K = \langle x_mx_n :m,n\in \mathbb{Z}_{\geq 0} \rangle.$$ The quotient $B$-algebra equals $R\oplus M$ where $M$ is a square-zero ideal that is the free, countably-generated $R$-module with basis $$\{ x_n : n\in \mathbb{Z}_{\geq 0} \}.$$ In particular, the element $x_0$ in this quotient algebra is not $t$-divisible in $M$. Thus, there is no $t$-divisible element of $B$ that maps to $x_0$ in the quotient algebra $A$. Therefore, there is no retraction $s$, and $A$ is not formally smooth.

Welcome new contributor. This is too long for a comment. Let $k$ be a field. Let $R$ be $k[[t]]$. Let $C$ be the $R$-algebra $$C=R[x_n:n\in \mathbb{Z}_{\geq 0}].$$ Let $J$ be the ideal in $C$, $$J=\langle x_n-tx_{n+1} :n\in \mathbb{Z}_{\geq 0} \rangle.$$ Let $A$ be the $R$-algebra $C/J$.

Since $A$ has no $t$-torsion, the $R$-module $A$ is $R$-flat. The fiber ring $A\otimes_R (R/t R)$ equals $R/tR$, which is smooth over $R/tR$. The fiber ring $A\otimes_R R[1/t]$ equals $R[1/t][x_0]$, which is smooth over $R[1/t]$.

Now consider the $R$-algebra $B=C/J^2$. Let $I$ be the image ideal in $B$, $$I=J/J^2.$$ This is a square-zero ideal whose quotient algebra equals $A$. Thus, if $A$ is formally smooth, there exists a retraction of $R$-algebras, $$s:A\to B.$$ In particular, $s(x_0)$ is a $t$-divsible element of $B$ that maps to $x_0$ in the quotient algebra $A$.

Consider the quotient of $B$ by the ideal $$K = \langle x_mx_n :m,n\in \mathbb{Z}_{\geq 0} \rangle.$$ The quotient $B$-algebra equals $R\oplus M$ where $M$ is a square-zero ideal that is the free, countably-generated $R$-module with basis $$\{ x_n : n\in \mathbb{Z}_{\geq 0} \}.$$ In particular, the element $x_0$ in this quotient algebra is not $t$-divisible in $M$. Thus, there is no $t$-divisible element of $B$ that maps to $x_0$ in the quotient algebra $A$. Therefore, there is no retraction $s$, and $A$ is not formally smooth.