Maybe this isn't very high concept, but I've always thought the "original sin" of 2 was that there's a integer which is a second root of unity, which doesn't happen for any other prime.
Why is this deep? Well, one way to think of it as this: in fields of characteristic p, pth roots of unity must all be trivial (and in general, taking pth roots is a bad idea), so fields of characteristic 2 are particularly incompatible with the integers, since they have to destroy -1.
