The answer to this question is no in general. Let $K$ be a field, and let $R=F\{s,t\ :\ st=s+t \}$, the non-unital algebra generated by the non-commuting variables $s,t$ subject to the single relation $st=s+t$.
Let $A$ be any unital $K$-algebra containing $R$. In $A$ we have $(1-s)(1-t)=1-s-t+st=1$. However, $(1-t)(1-s)=1-t-s+ts\neq 1$ since $ts\neq s+t$. Thus $(1-s)$ is only right invertible, and hence $A\setminus R$ does not consist only of units.
Edited to add: An easier example, which has the added benefit of being commutative is taking $R$ to be the non-unital ring generated by commuting variables $x,y$ subject only to the additional relation $x^2=x$. The element $1-x\in A\setminus R$ is not unit, since it is a non-trivial idempotent.