Whenever I've seen the definition of the classifying space of a category, the category is always specified to be small. I understand the definition well enough for my purposes (I think), but it occurred to me today, why small?

Is there a reason we only take nerves of small categories? Does the definition fail if the objects are too big?