Let $A$ be an abelian variety over an algebraically closed field $k$.

Let $\phi:A\to A$ be an étale isogeny (over $k$). Suppose that the set $\cup_{r\geq 0}({\rm ker}\,\phi^{\circ r})(k)$ is Zariski dense in $A$. Here $\phi^{\circ r}:=\phi\circ\dots\circ\phi$ ($r$-times).

Now let $\lambda:A\to A$ be an endomorphism and suppose that for all $r\geq 0$, we have $$\lambda(({\rm ker}\,\phi^{\circ r})(k))\subseteq ({\rm ker}\,\phi^{\circ r})(k)\,\,\,\,(*).$$

Does it follow that we have $$\phi^{-1}\circ\lambda\circ \phi=\lambda\circ c$$ in ${\rm End}(A)_{\bf Q}$ for some $c\in{\rm End}(A)_{\bf Q}$ which commutes with $\phi$? Or more generally, what relation in ${\rm End}(A)_{\bf Q}$ between $\lambda$ and $\phi$ does $(*)$ imply?

One may of course ask the same question for any isogeny (not just étale) and formulate a more general condition involving finite group schemes but the case above is the case I am interested in.