Proposition 6.2 of Formality of DG algebras (after Kaledin) by Lunts reads (with a few additions to clarify notation):
Let $k$ be a field of characteristic 0. Let $A$ be an $A_\infty$ algebra over a commutative $k$-algebra $R$ which is a finite as an $R$-module. Let $R\to Q$ be a homomorphism of commutative rings. Assume that $Q$ is a faithfully flat $R$-module. Then $A$ is formal if and only if the $A_\infty$ $Q$-algebra $A\otimes_RQ$ is formal.
Does this theorem fail for a DG-algebra of positive characteristic? Is there some additional hypothesis I can add to fix it? Specifically, would it help to know that $A$ was actually the specialization of an algebra $A_{\mathbb{Z}}$, which is formal after base change to a characteristic 0 field?