By standard results (in fin dim, over an alg closed field of zero char),
- all cocommutative HAs are group algebras (for some finite group),
- all commutative HAs are duals of group HAs (for some finite group)
(see for example: About the classification of commutative and of cocommutative, fin. dim. Hopf algebras)
More generally, if we consider fd hopf algebras, over a field of char zero then, by the Larson-Radford theorem, semicimplicity and cosemisimplicity are equivalent notions and they both result either from commutativity or cocommutativity.
Consequently, since you are looking to construct noncommutative, noncocommutative HAs, going beyond the semisimple/cosemisimple case would guarantee that the non-(co)semisimple HAs will also be non-(co)commutative. So i guess a natural class of examples would be pointed HAs.
The Taft algebras constitute a standard class of non-commutative, non-cocommutative examples. Sweddler's hopf algebra (mentioned in the OP) is a special case of them.
There are various results on the classification of f.d. hopf algebras which have been proved during the last couple of decades (with still lots of open problems) that can be of use for the purposes of the OP. Assuming that we are speaking about an algebraically closed field, of zero char, one should take into account both (co)semisimple and non-(co)semisimple cases:
- First note that all prime dimensions should be excluded, since by Zhu's theorem (also mentioned in the OP), these are all group hopf algebras of cyclic groups of prime order. Hence, these are commutative, cocommutative, semisimple and cosemisimple.
- $dim=6$: a Hopf algebra of dimension 6 is either commutative or cocommutative. These are all semisimple.
- $dim=8$: There are $7$ group algebras (up to isomorphism) and their duals (which are either comm or cocom).
There is a unique (up to isomorphisms) semisimple Hopf algebra of dimension 8 that is neither commutative nor cocommutative. This is Kac's hopf algebra. See: Semisimple Hopf algebras of dimension 6, 8, A. Masuoka, Israel J. of Math. v. 92, p. 361–373, (1995). It interesting to note that dim $8$ is the smallest dimension for which (co)semisimplicity does not imply (co)commutativity.
The non-semisimple case for dim 8 is more interesting though: It includes $6$ more isomorphism classes of HAs. These are either pointed or their duals are pointed. See: A Survey of Hopf Algebras of Low Dimension, M. Beattie, Acta Applicandae Mathematicae, v. 108, p.19–31, (2009) for their explicit description. - $dim=9$: All semisimple HAs of dim $p^2$, $p=2,3$, are trivial (i.e. group algebras or their duals). In the non-semisimple case we have $p-1$ (pointed) Taft algebras.
- $dim=10$: the semisimple cases are trivial, i.e. we have only group algebras and their duals. This is a general result for $pq$-dim HAs: Semisimple Hopf Algebras of Dimension $pq$ Are Trivial, J. of Algebra, v. 210, 2, 1998, p. 664-669. Τhere are no non-semisimple HAs of dim $10$.
See also the table included in p. 23 of Classifying Hopf algebras of a given dimension, arXiv:1206.6529v2 [math.QA] and the relevant literature included there. Following it, you can extend the above classification for higher dimensional HAs.
Edit: A particularly interesting article is Hopf Algebras of Low Dimension, Stefan, Journal of Algebra 211, 343 361 (1999). There, the author classifies all isomorphism types of Hopf algebras of dimension $\leq 11$ over an algebraically closed field of characteristic 0.