As you say, given a symmetric monoidal category $\mathcal C$ enriched in abelian groups, the words "Lie algebra object in $\mathcal C$" and "associative algebra object in $\mathcal C$" make sense. (Actually, the latter does not depend on the symmetric structure nor the ab-gp enrichment.) In particular, there are natural categories $\text{LieAlg}_{\mathcal C}$ and $\text{AssocAlg}_{\mathcal C}$ — it makes sense to say whether an arrow in $\mathcal C$ between Lie/associative algebra objects is a homomorphism — and there is a natural "forgetful" functor from associative algebra objects to Lie algebra objects. If this functor has a left adjoint, said adjoint deserves to be called "free" or "universal enveloping" (but see below).
Of course, you are not guaranteed such an adjoint. For example, in the category of finite-dimensional vector spaces you cannot build (most) UEAs. You can see this very explicitly: working over characteristic $0$, the Lie algebra $\mathfrak{sl}(2)$ acts faithfully and transitively on representations of arbitrary dimension, and so $U(\mathfrak{sl}(2))$ cannot be finite-dimensional.
The minimum extra structure that I know of to guarantee the existence of a left-adjoint to $\text{Forget}: \text{AssocAlg}_{\mathcal C} \to \text{LieAlg}_{\mathcal C}$ is:
- Existence of arbitrary countable direct sums in $\mathcal C$.
- Existence of cokernels in $\mathcal C$.
If you have these, then you can do the usual construction to define $U\mathfrak g$.
If you are working in a category in which all hom sets are vector spaces over $\mathbb Q$, then you can also define $U\mathfrak g$ as a deformation of the symmetric algebra $S\mathfrak g$, provided this symmetric algebra exists. Namely, pretend for a moment that our category is just the usual category of $\mathbb K$-vector spaces for $\mathbb K$ a field of characteristic $0$. Then there is a "symmetrization" map $S\mathfrak g \to U\mathfrak g$ given on monomials by $x_1\cdots x_n \mapsto \frac1{n!} \sum_{\sigma \in S_n} x_{\sigma(1)}\cdots x_{\sigma(n)}$, where $S_n$ is the symmetric group in $n$ letters. This is a (filtered) vector space isomorphism (and also a coalgebra isomorphism, and also a $\mathfrak g$-module isomorphism), and so you can use it to pull back the algebra structure on $U\mathfrak g$ to one on $S\mathfrak g$, which you should think of as some sort of "star product".
So do this in $\mathbb K$-vector spaces, and then interpret the formulas on $\mathcal C$.
For details, and in particular for an explicit formula for the star product in terms of the usual monomial basis on $S\mathfrak g$, see:
- Deligne, Pierre; Morgan, John W.
Notes on supersymmetry (following Joseph Bernstein). Quantum fields and strings: a course for mathematicians, Vol. 1, 2 (Princeton, NJ, 1996/1997), 41--97, Amer. Math. Soc., Providence, RI, 1999. MR1701597
But I see no conditions weaker than 1–2 above to guarantee the existence of the symmetric algebra.
Finally, I should mention that in general, even if $\text{Forget}$ has a left adjoint $U$, it does not necessarily deserve to be called the "universal enveloping algebra". Namely, simply by being an adjoint, there is a canonical Lie algebra map $\mathfrak g \to U\mathfrak g$. For $U\mathfrak g$ to "envelop" $\mathfrak g$, this map should be a monomorphism in $\mathcal C$.
The following example is due to:
- Cohn, P. M.
A remark on the Birkhoff-Witt theorem.
J. London Math. Soc. 38 1963 197--203. MR0148717
Let $\mathbb K$ be a field of characteristic $p \neq 0$, and consider the free associative (noncommutative) algebra $\mathbb K \langle x,y\rangle$. Then $\Lambda_p(x,y) \overset{\rm def}= (x+y)^p - x^p - y^p$ is a non-zero Lie polynomial — it is a sum of compositions of brackets. For example, $\Lambda_2(x,y) = [x,y]$ and $\Lambda_3(x,y) = [x,[x,y]] + [y,[y,x]]$.
Let $R = \mathbb K[\alpha,\beta,\gamma]/(0 = \alpha^p = \beta^p = \gamma^p)$; it is a commutative ring. Let $\mathcal C = R\text{-mod}$ be the category of $R$-modules, with the usual symmetric tensor structure $\otimes_R$. Let $\mathfrak f_3$ be the free Lie algebra in $\mathcal C$, with the generators $x,y,z$, and let $\mathfrak g = \mathfrak f_3 / (\alpha x = \beta y + \gamma z)$.
Then $\Lambda_p(\beta y,\gamma z)$ is non-zero in $\mathfrak g$, but is $0$ in $U\mathfrak g$. Hence, internal to $\mathcal C$, $\mathfrak g$ does not embed into its universal enveloping algebra. (Of course, it does if we were just working over $\mathbb K$, as then the original PBW proof applies. And we always have an embedding in characteristic $0$, as there we can define $U\mathfrak g$ as a deformation of $S\mathfrak g$.)