Assume $J$ is not projective as a right $A$-module, or equivalently that $A$ is not hereditary. Then $\mbox{Ext}^1_A(J,J) \neq 0$ and it is possible to write down a somewhat explicit non-split extension of direct summands of $J$ as follows.
Let $e_1, \ldots, e_n$ be a complete set of pairwise orthogonal primitive idempotents in $A$, and assume $A$ is basic. Since $J = \oplus_i e_iJ$, by assumption $e_iJ$ is not projective for some $i$. If we decompose $e_iJ = P \oplus M$ where $P$ is projective and $M$ has no projective direct summands, then by mapping $M$ onto a simple module $S_j = e_jA/e_jJ$ in its top, we get an epimorphism $f : e_i J \to S_j$ that cannot factor through a projective module.
Thus the pull-back of the short exact sequence
$$0 \to e_j J \to e_jA \to S_j \to 0$$
along $f: e_i J \to S_j$, yields a non-split short exact sequence
$$0 \to e_jJ \to X \to e_iJ \to 0.$$
Moreover, using the snake lemma, the induced map $X \to e_jA$ is onto and thus splits, showing that $X \cong e_j A \oplus \ker(f)$, and $\ker(f)$ is a maximal submodule of $e_i J$. It is not hard to see what the maps are either. The maps $e_jJ \to e_jA$ and $\ker(f) \to e_iJ$ are just the inclusions, while the maps $e_jJ \to \ker(f)$ and $e_jA \to e_iJ$ would typically be induced by multiplication by some arrow from $i$ to $j$ in the quiver of $A$.

We can now add a split short exact sequence to get a non-split extension of $J$ by $J$ of the form
$$0 \to J \to e_jA \oplus \ker(f) \oplus (1-e_j)J \oplus (1-e_i)J \to J \to 0.$$