If a faithfully flat extension of dg/A_$\infty$-algebra is formal, is the original algebra formal (over positive characteristic)?

Proposition 6.2 of Formality of DG algebras (after Kaledin) by Lunts reads (with a few additions to clarify notation):

Let $k$ be a field of characteristic 0. Let $A$ be an $A_\infty$ algebra over a commutative $k$-algebra $R$ which is a finite as an $R$-module. Let $R\to Q$ be a homomorphism of commutative rings. Assume that $Q$ is a faithfully flat $R$-module. Then $A$ is formal if and only if the $A_\infty$ $Q$-algebra $A\otimes_RQ$ is formal.

Does this theorem fail for a DG-algebra of positive characteristic? Is there some additional hypothesis I can add to fix it? Specifically, would it help to know that $A$ was actually the specialization of an algebra $A_{\mathbb{Z}}$, which is formal after base change to a characteristic 0 field?

I'm not sure off hand what the situation is for $A_\infty$-algebras, but for $\mathbb{E}_\infty$-algebras it's worth noting that in many cases generic formality does not imply formality at each fiber. For example, consider the cochain algebra $C^*(X,\mathbb{Z})$. The rationalization $C^*(X,\mathbb{Q})$ can often be formal, as is the case for $X$ compact Kähler by the theorem of Deligne-Griffiths-Morgan-Sullivan. However, $C^*(X,\mathbb{F}_p)$ is not formal if it is non-trivial. In fact, it is not even equivalent to a cdga at all! This follows from the the fact that Steenrod operation $Sq^0$ is the identity on cohomology of a space, but acts as zero on $H^i(A)$ for $i\neq 0$ if $A$ is a cdga over $\mathbb{F}_p$. (I learned this originally from Mike Mandell.)
Of course, this does not actually answer your question. The reason that Kaledin's theorem works, as explained by Lunts, is that there is an obstruction to formality in a Hochschild cohomology group that behaves well under base extension. Unfortunately, the definition seems to use the Campbell-Hausdorff formula, so it's not clear to me what happens at all in characteristic $p$.
Yanki Lekili asked me a potentially related question: are the cochain algebras $C^*(G,\mathbb{F}_p)$ formal as dgas not as $\mathbb{E}_\infty$-algebras for compact Lie groups $G$?