There is a general bit of category theory that was applied to
ring spectra in EKMM ([83] on my website) and I'll refer
to that for details. Unless I am missing something, the
discussion surely specializes just as well to dg algebras over a
commutative ring $R$. I'll outline the recipe it gives
for constructing all colimits of dg $R$-algebras as reflexive
coequalizers in the category $Ch_R$ of chain complexes over $R$.

The free graded $R$-algebra functor induces a free dg $R$-algebra
functor on $Ch_R$. That gives a monad $T$ on $Ch_R$ whose algebras
are the dg $R$-algebras. This monad preserves reflexive coequalizers
by Prop. 7.2, p. 47. Therefore, by Lemma 6.6, p. 46, if $g\colon B\to C$
is a reflexive coequalizer of maps $e,f\colon A\to B$ in $Ch_R$ such that
$A$ and $B$ are $T$-algebras and $e$ and $f$ are maps of $T$ algebras, then
$C$ has a unique structure of $T$-algebra such that $g$ is a
map of $T$-algebras, and $g$ is the coequalizer of $e$ and $f$
in the category of $T$-algebras. Now all colimits in the
category of $T$-algebras are constructed from just such
reflexive coequalizer diagrams in $Ch_R$, as shown in the
proof that the category of $T$-algebras is cocomplete given in
Prop. 7.4, p. 49.