The goal of the following answer is to drop bicommutativity from Neil Strickland's answer.

You told us already what is an unbiased associative algebra. They certainly form an unbiased monoidal category $\text{UnbAlg}$. Let me define an *unbiased bialgebra* to be an unbiased associative algebra object in $\text{UnbAlg}^{op}$. Note that it is most natural to talk about unbiased algebra objects in an unbiased monoidal category.

Let me unpack this definition, following the notation from Neil's answer. Suppose $A$ and $B$ are finite *totally ordered* sets and $p : A \to B$. Then saying that $H$ is an associative algebra means I can assign $\mu_p : H^{\otimes A} \to H^{\otimes B}$. Now suppose I also have $q : X \to Y$. The comultiplication is an algebra homomorphism $\psi_q : H^{\otimes Y} \to H^{\otimes X}$. What does that mean? It means that the diagram
$$ \begin{CD}
(H^{\otimes Y})^{\otimes A} @>{\mu_p^{\otimes Y}}>> (H^{\otimes Y})^{\otimes B} \\
@V{\psi_q^{\otimes A}}VV @VV{\psi_q^{\otimes B}}V\\
(H^{\otimes X})^{\otimes A} @>>{\mu_p^{\otimes X}}> (H^{\otimes X})^{\otimes B}
\end{CD}$$
should commute. Note that these are really maps between $H^{\text{rectangle}}$. So I am reminded of Haugseng's version of the Morita category of $E_2$ algebras...

Niel already addressed the issue of the antipode in the bicommutative case. I cannot improve on his answer in the unpacked definition, and I don't know a version that meshes well with the idea that bialgebras are algebra objects in $\text{Alg}^{op}$. The problem, of course, is that the antipode map is neither an algebra nor coalgebra homomorphism in the noncommutative noncocommutative case.

Probably that is a hint that the unbiased version of antipode does not address just the antipode, but all of its powers. Indeed, in the noncommutative noncocommutative case, the antipode may not be an involution. Rather, as Neil suggests, you should write down some lift of compositions of $\mu$s and $\psi$s all of which you ask to be invertible.

Let me recall that a properad is like an operad, except you can have many-to-many vertices, hence you can compose in graphs. (The graphs are directed, and must not contain directed cycles.) The restriction is that in a properad, the composition graphs should be connected, whereas in a prop composition graphs may be disconnected. Properads were invented by Vallette. There is obviously a prop for bialgebras: I say "obviously" because the bialgebra axiom (2-1-2 = 2-4-2) only requires connected graphs.

This same bialgebra axiom allows you to normally order (I mean, it provides a PBW basis for) any composition in the Hopf properad: you can do all comultiplications first, and then all multiplications.

Let's suppose you have a PBW-basis entry, meaning all comultiplications come before all multiplications, and the diagram is connected. Let me say it is a *ladder* if it has the same number of inputs and outputs and is a tree. Then I think $H$ is Hopf — meaning has an antipode — exactly when for every ladder, the corresponding linear map $H^{\otimes n} \to H^{\otimes n}$ is an isomorphism? I could be wrong...