From the classification of closed surfaces, it follows that any closed surface can be obtained by gluing discs and pants along their boundaries. I wonder whether a similar statement holds in higher dimensions.

Let $n\geq 3$. Does there exist a finite collection $\mathcal{C}$ of $n$-manifolds with boundary such that any closed $n$-manifold can be obtained from manifolds of $\mathcal{C}$ by identifying connected components of their boundaries?

I think a negative answer is known, but I am looking for precise references.

PS: If needed, you can consider that the manifolds here are smooth.

PSS: I asked the same question on math.stackexange.