The notion of vertex algebra, like any reasonable algebraic notion, makes sense inside any (sufficiently linear) symmetric monoidal category. The standard pictures of the operator product, however, suggest that symmetry is more than is needed: one should be able to formulate the axioms of "vertex algebra" internal to any (sufficiently linear) braided monoidal category. Has this been done in the literature? Are there standard examples?
For vertex operator algebras, I expect you need your braided monoidal category to be balanced.