The category of left exact functors $E\colon\mathcal A_p^{op}\to\mathcal Ab$ is equivalent to the category of $p$-primary torsion abelian groups $T$. To a $p$-primary torsion abelian group $T$ one assigns the contravariant functor $E_T$ taking a finite abelian $p$-group $A$ to the abelian group $\operatorname{Hom}_{\mathbb Z}(A,T)$. To construct the inverse functor, consider the projective system of finite abelian groups$p$-groups $\mathbb Z/p\mathbb Z \leftarrow \mathbb Z/p^2\mathbb Z\leftarrow \mathbb Z/p^3\mathbb Z\leftarrow\dotsb$ and apply the contravariant functor $E$ to it. The $p$-primary torsion abelian group $T_E$ corresponding to $E$ is the inductive limit of the sequence $E(\mathbb Z/p\mathbb Z)\to E(\mathbb Z/p^2\mathbb Z)\to E(\mathbb Z/p^3\mathbb Z)\to\dotsb$. From the left exactness property of the functor $E$ one can see that the group $E_m=E(\mathbb Z/p^m\mathbb Z$) is identified with the subgroup of elements annihilated by $p^m$ in the group $E_n$, for every $n\ge m$.
The categories of $p$-separated $p$-complete abelian groups and $x$-separated $x$-complete $k[x]$-modules are complete and cocomplete. In fact, they are locally $\aleph_1$-presentable, being reflective and closed under $\aleph_1$-filtered colimits as full subcategories in the locally $\aleph_1$-presentable abelian categories of Ext-$p$-complete abelian groups and Ext-$x$-complete $k[x]$-modules (respectively). So, in particular, all morphisms in these two categories (of right exact functors) have kernels and cokernels. Still, these two categories are not abelian.
The counterexample showing that the categories of $p$-separated $p$-complete abelian groups and $x$-separated $x$-complete $k[x]$-modules are not abelian is now well-known. It has been rediscovered and discussed by many authors, including Example 2.5 in A.-M. Simon, "Approximations of complete modules by complete big Cohen-Macaulay modules over a Cohen-Macaulay local ring", Algebras and Represent. Theory 12, 2009, and Example 3.20 in A. Yekutieli, "On flatness and completion infor infinitely generated modules over Noetheriannoetherian rings", Communic. in Algebra 39, 2010, https://arxiv.org/abs/0902.4378 .