This is not accurate as far as discrete groups are concerned. First of all it is an open question, called Kaplansky's direct finiteness conjecture, whether every group ring over a field has the property that $ab=1$ implies $ba=1$ (i.e. is directly finite). The best result to date is due to Elek and Szabo who proved if $G$ is a sofic group (this includes Gardam's example from the unit conjecture), then $KG$ is directly finite for any field $K$. It is an open question whether every group is sofic.
It's also not true that $C^*$-algebras are needed in characteristic 0 although all known proofs to the best of my knowledge use at least some "analysis". Passman gave a proof (you can find it in his book The Algebraic Theory of Group Rings) using just basic epsilon-delta stuff inside of $\mathbb CG$ without going to the completion. The journal reference is Passman, D. S., Idempotents in group rings.
Proc. Amer. Math. Soc. 28 (1971), 371–374.
To give some details, the key thing need to prove that $\mathbb CG$ is directly finite is to show that the trace of any nonzero idempotent is positive. The trace here is the coefficient of $1$. Kaplansky uses that the group von Neumann algebra has a faithful trace extending this trace (it is enough that the reduced $C^*$-algebra have such a trace). Then Kaplansky uses that in a $C^*$-algebra, every idempotent is equivalent to projection (i.e., $e=ab$ and $p=ba$ with $p$ a projection). Since a faithful trace is positive on nonzero projections, it must be positive on nonzero idempotents. The argument uses that in a $C^*$-algebra, $1+aa^*$ is always invertible (by spectral theory). The group algebra is a $*$-algebra, but it is not complete and $1+aa^*$ is not always (or even usually) invertible. Passman gives a direct proof that the trace of any nonzero idempotent of $\mathbb CG$ is positive via elementary epsilon-delta arguments.
Added. In fact if a group has a directly finite algebra over all finite fields, then it has a directly finite algebra over every field and hence for sofic groups, operator algebras and analysis are not needed at all (the proof for sofic groups is not analytic). If all groups are sofic, analysis is not needed.
The reduction to finite fields is standard. If $a,b\in KG$ with $ab=1$ but $ba\neq 1$, then there are only finitely many coefficients appearing in $a,b$ and so we can find a finitely generated (over $\mathbb Z$) subring $R$ of $K$ with $RG$ containing $a,b$. By a version of nullstellensatz, since $R$ is a finitely generated integral domain the intersection of the maximal ideals of $R$ is trivial and each quotient of $R$ by a maximal ideal is a finite field. Since $0\neq ab-ba$, we can find a finite field $F$ which is an image of $R$ in which the image of $ab-ba$ under $RG\to FG$ is nonzero. Then $FG$ is not directly finite.