I think the answer to your question is "yes". Toen-Vezzosi go over this in Proposition 2.4.15, but here is some version of why.
Away from characteristic zero, there's a sharp difference between being a "free" algebra (meaning, having some kind of universal mapping property) and looking like a polynomial algebra. This is because basically every construction has to be replaced with the derived version in order for them to give sensible answers.
For example, in characteristic zero you might take a vector space $V$ over $\mathbb{Q}$ and form the "free" algebra
$$
Sym(V) = \bigoplus_{n \geq 0} Sym^k(V)
$$
and this has the property that maps of commutative ring objects $Sym(V) \to A$ are the same as maps $V \to A$ of underlying objects. This still works even if $V$ is a chain complex.
However, the symmetric power functors aren't well-behaved integrally: there are maps of chain complexes $C \to D$ which are weak equivalences such that $Sym^k(C) \to Sym^k(D)$ isn't a weak equivalence. (These examples aren't particularly hard to find, either.) This means that we have to take the symmetric power
$$
Sym^k(V) = (V^{\otimes k})_{\Sigma_k}
$$
and replace it with the derived version: derived tensor product over whatever your base is, and derived functors of $\Sigma_k$-coinvariants.
Toen-Vezzosi mention the example which they call $\mathbb{F}_p[T]$, which is free on a 1-dimensional vector space over $\mathbb{F}_p$. Here, the derived functors of tensor product don't intrude, but the derived functors of coinvariants do, and they contribute a large amount (namely, the homology of the symmetric groups). In degree zero you're just getting a polynomial algebra on a single generator, but there is extra stuff in positive (homological) degree coming from the Dyer-Lashof operations.
EDIT: Sorry, this object is the "smooth" object.
The thing that looks like a "polynomial" object is the monoid algebra $\mathbb F_p[\mathbb N]$. On homology groups this looks like a polynomial algebra, but it doesn't have simple mapping properties that connect with the notion of smoothness. In particular, they give a description of the derived cotangent complex (which is some measure of how hard it is to build this object out of "free" objects) and it's rather large (certainly not concentrated in degree 0).
