This question is going to be extremely vague.

It seems that wherever I go (especially about Grothendieck's circle of ideas) the higher-dimensional analogue of a curve minus a finite number of points is a scheme minus a normal crossing divisor.

Why is that? What's so special about a normal crossing divisor that it simulates a curve minus a finite number of points better?