Generalization of Tamarkin’s ARO 1993, final round, problem 10/8: part II

Let us use the notations of my previous question about Tamarkin's problem.

Let $\ell\in\left\lbrace 0,1,...,p\right\rbrace$.

An element $f\in \mathbb Z^{\mathbb Z}$ is said to be $\ell$-average-integral if it satisfies $F_p^kf\in\mathbb Z^{\mathbb Z}$ for every $k\in\left\lbrace 0,1,...,\ell\right\rbrace$.

For any $f\in\mathbb R^{\mathbb Z}$, define $f^p\in\mathbb R^{\mathbb Z}$ by

$f^p\left(n\right)=\left(f\left(n\right)\right)^p$ for every $n\in\mathbb Z$.

Conjecture: If $f\in \mathbb Z^{\mathbb Z}$ is $\ell$-average-integral, then so is $f^p$.

This is motivated by zeb's results, some of which I have yet to properly understand. There is a counterexample with $\ell=p+1$ for every prime $p > 3$, whence the condition $\ell\in\left\lbrace 0,1,...,p\right\rbrace$. Is there a counterexample with $\ell=p$ as well? And if there is, are there counterexamples with smaller $\ell$?

Note that the cases $\ell=0$ and $\ell=1$ are trivial, and I believe I have checked the $\ell=2$ case to be true by hand.

-