We give a counterexample to the following statement that came up in the question/comments:
Any formally smooth flat map $A \to B$ is a filtered colimit of smooth maps.
In the example, $A$ is noetherian (even a field), but $B$ is not. The example takes place in characteristic $p > 0$.
Say $A = \mathbb{F}_p$ is a finite field with $p$ elements. Let $B_0 = A[x,y]/(xy)$ be the co-ordinate ring of the union of the two axes in the plane, and let $B = B_{0,\mathrm{perf}}$ be the perfection of $B_0$, so $B = \mathrm{colim}_i B_i$ is a filtered colimit of copies of $B_i := B_0$ along Frobenius maps. Note that $B_0$ is not a domain. Now:
Claim: The map $A \to B$ is formally smooth and flat, but not a filtered colimit of smooth $A$-algebras.
The flatness is clear as $A$ is a field. Also, any map between perfect rings is formally smooth by an elementary lifting argument. (In fact, $L_{B/A} \simeq 0$.) Now assume that we can write $B = \mathrm{colim}_j C_j$ as a filtered colimit of smooth $A$-algebras $C_j$; this will lead to a contradiction.
As each $C_j$ and $B_i$ is finitely presented, by comparing the two filtered inductive systems $\{B_i\}$ and $\{C_j\}$, one finds a factorisation
$$ B_i \to C_j \to B_{i'} $$
of the structure map $B_i \to B_{i'}$ for $i' \gg i$ and suitable $j$. As the transition maps in $\{B_i\}$ are Frobenius, we conclude: there is a smooth $A$-algebra $C$ and a factorisation
$$B_0 \stackrel{a}{\to} C \stackrel{b}{\to} B_0$$
with $b \circ a = \mathrm{Frob}_{B_0}^n$ for some $n \geq 0$. As the target $B_0$ is connected, after replacing $C$ by a connected component, we may assume $C$ is also connected, and thus a domain (by smoothness). The existence of the above factorisation shows that $a$ is injective (as the composite $b \circ a$ is injective), so $B_0$ is also a domain, which is nonsense.