Generally I think that in fact you're looking at "small" vs. "big", rather than finite vs. infinite. In each field there is a different notion of what small means: in category theory, for example, we tend to distinguish categories with a set of objects rather than a class. For manifolds we are often looking at only finite dimensional anyways, so we care about compactness vs. non-compactness.

But the big thing is that small things behave very differently from big things, quite often because we can count/classify the small things but we can't the big things. So we try to keep ourselves to the cases that we know and rule out the things we don't, because otherwise we get very lost.

And as a last note I want to point to the Eilenberg-Mazur swindle, which shows that infinite sums generally are not associative. Very weird things happen with infinity: $\infty+1=\infty=2\infty$, for example. This means that in order to deal with infinite (big) things we need different tools than when dealing with finite (small) things.