$\newcommand{\cA}{\mathcal{A}}\newcommand{\cB}{\mathcal{B}}\newcommand{\bZ}{\mathbb{Z}}$No, that does not imply that $\cA$ splits over $R$. In fact, if $\cA_1=\cA\times_R R/p$ is isogenous to a product of elliptic curves then all $\cA_n=\cA\times_R R/p^n$ are isogenous to products of elliptic curves:
Lemma. Let $S'\twoheadrightarrow S$ is a surjection of $\bZ/p^n$-algebras with nilpotent kernel. If $\cA,\cB$ are abelian schemes over $S'$ then for any morphism $f:\cA\times_{S'}S\to\cB\times_{S'}S$ of abelian schemes over $S$ there exsists a morphism $g:\cA\to\cB$ over $S'$ reducing to $p^Nf$ for a certain $N$.
Proof. The analogous statement for $p$-divisible groups is a part of Drinfeld's rigidity lemma for quasi-isogenies and the statement for abelian schemes follows from Serre-Tate theorem(Lemma 1.1.3 and Theorem 1.2.1 in Katz's 'Serre-Tate local moduli').
In fact, this particular consequence can be also derived by a more direct argument: we can define the desired $g$ on the level of functors of points. There exists $N$ such that for any $S'$-algebra $T$ the kernel of the surjection $\cB(T)\to \cB_{S}(T\otimes_{S'}S)$ is annihilated by $p^N$. For a section $x\in\cA(T)$ define then $g(x)$ as $p^N\widetilde{f(\overline{x})}$ where $\widetilde{\cdot}$ denotes an arbitrary lifting along the surjection $\cB(T)\to\cB_S(T\otimes_{S'}S)$, the result does not depend on the choice. $\square$
Applying the lemma to the thickening $R/p^n\to R/p$ and the abelian schemes $\cA_n$ and $\widetilde{E}_1\times\widetilde{E'_1}$ where $\widetilde{E}_1,\widetilde{E'_1}$ are arbitrary lifts of the elliptic curves $E_1,E_1'$ gives the claim. The key here is, of course, that we are not bounding the degree of a splitting isogeny.
So, it remains to give an example of a simple abelian surface $\cA$ over $R$ with reduction isogenous to a product of elliptic curves. Here is one way to do this: pick a quaternion algebra $D$ over $\mathbb{Q}$ that is non-split at $p$ but is split over $\mathbb{R}$ and an abelian surface $\cA/R$ with $End_R(\cA)\otimes_{\bZ}\mathbb{Q}\simeq D$ (abelian surfaces with an action of an order in $D$ are parametrized by a Shimura curve and we just need to choose a general enough point on it so that the endomorphism algebra is not larger than $D$). Because of the assumption on $p$ the reduction of $\cA$ modulo $p$ is supersingular and is isogenous to a product of two supersingular elliptic curves(see e.g. Proposition 2 in Chapter III of Boutout-Carayol's 'Uniformisation p-adique des courbes de Shimura').