In the non-commutative situation, it is called Groebner-Shirshov basis. See, for example, the survey paper "[Groebner-Shirshov Bases: Some New Results][1]" by L. A. Bokut and Yuqun Chen, and references there. The case of universal enveloping algebra was considered by Bergman in G. M. Bergman, [The diamond lemma for ring theory][2], Adv. in Math., 29, 178-218 (1978). I do not know about representation theory, but certainly this is one of the main tools of studying finitely presented rings and algebras over a field. The problem, of course, is that a finite Groebner-Shirshov basis (unlike Groebner basis in the commutative case) does not always exist. [1]: https://arxiv.org/abs/0804.1344 "Proceedings of the 2nd international congress of Algebras and Combinatorics, World Scientific, 2008, 35-56, doi:10.1142/9789812790019_0003. zbMATH review at https://zbmath.org/1207.16023" [2]: https://doi.org/10.1016/0001-8708(78)90010-5 "zbMATH review at https://zbmath.org/0326.16019"