In the introduction to his paper "Towards a non-abelian $p$-adic Hodge theory", Olsson says that for any ringed topos $(\mathcal{T},\mathcal{O})$ with $\mathcal{O}$ a sheaf of $\mathbb{Q}$-algebras, the category $\mathrm{dga}_{\mathcal{O}}$ of $\mathcal{O}$-dga's has a model category structure, where the weak equivalences are quasi-isomorphisms, and the fibrations are surjections with level wise injective kernel (injective as objects in the category of $\mathcal{O}$-modules).
Now, it seems to me the proof of this fact goes something along the following lines. Since the category of $\mathcal{O}$-modules has enough injectives, then the category of positively graded chain complexes has a similarly defined model category structure. One takes generating sets cofibrations and acyclic cofibrations in this model category of chain complexes, and one applies the 'free algebra functor' from complexes to dga's and the small object argument to get generating sets of cofibrations and acyclic cofibrations in the category of dga's.
My question is the following.
Do we really need to restrict to $\mathbb{Q}$-algebras here? Or will this argument work for any ringed topos $(\mathcal{T},\mathcal{O})$? For example, will the above definitions of weak equivalences and fibrations define a model category structure on the category of sheaves of $\mathbb{Z}/\ell^n$-modules in the étale topos of some scheme?
I can't see where the argument breaks down, but I may not have understood it well enough.