I'm afraid this is not very close to the case that you say you're most interested in ... maybe you want $F$ to preserve products?

But let $R=k[x]/(x^2)$, and let $\mathscr{A}$ be the category $\operatorname{Mod}R$ of $R$-modules. Let $\mathscr{F}$ be the Serre subcategory of finitely generated modules, $\mathscr{A}/\mathscr{F}$ the quotient category, and $F:\mathscr{A}\to\operatorname{Mod}k$ the functor $\operatorname{Hom}_{\mathscr{A}/\mathscr{F}}(V,-)$, where $V$ is an infinite direct sum of copies of $R/Rx$.

Let $C^\bullet$ be the complex with $R/Rx$ in every degree, and zero differentials. Then $Z^i(C^\bullet)$ and $B^i(C^\bullet)$ have injective resolutions by objects of $\mathscr{F}$ for all $i$, and are therefore "totally $F$-acyclic".

But $C^\bullet$ has a $K$-injective resolution $J^\bullet$ that is the product of all the shifts $\{I^\bullet[t]\mid t\in\mathbb{Z}\}$ of a minimal injective resolution $I^\bullet$ of $R/Rx$, and $F(J^\bullet)$ has zero differentials, but is nonzero in every degree, so $RF(C^\bullet)\neq0$