For the version of the question about finite products and coproducts:
Any meet-semilattice that does not have joins. For example, a linear order with no least element (which has no empty join).
Take any object $A$ in a category with all finite products and consider the full subcategory whose objects are all finite powers of $A$. This will usually fail to have coproducts. For example, the category of all finite sets whose cardinalities are powers of $2$.
For the version of the question about small products and coproducts:
The second example generalizes easily to arbitrary powers. For example, the category of all sets of cardinality $2^\kappa$ for arbitrary cardinals $\kappa$.
I originally claimed that the first example above generalizes easily as well, to complete meet-semilattices, but Max New correctly points out in the commentscomments that every complete meet-semilattice is a lattice. However, as Tim Campion also correctly points out in the commentscomments, a large semilattice may have all small meets without having all small joins. For example, the dual of the join-semilattice of all small subsets of a large set (which has no bottom element, hence no empty join).
Yet another naturalish example is the category of non-empty sets, which has all products but no initial object, hence no empty coproduct.