Why do Groups and Abelian Groups feel so different? Groups are naturally "the symmetries of an object".  To me, the group axioms are just a way of codifying what the symmetries of an object can be so we can study it abstractly.
However, this heuristic breaks down in the case of many abelian groups.  Abelian groups more often arise as a "receptacle for addition".  What I mean is that they are more intuitively counting combinations of some generating elements.  See for example: solution spaces to linear equations, the underlying group of rings and modules, (co)homology groups.  They rarely act naturally on anything except themselves, which seems like a copout.
It bugs me that these two intuitions are so far from each other, even though the underlying axioms differ by a single, deceptively-mild assumption.  Is there a way to reconcile these two perspectives?  Do abelian groups satisfy the group axioms by accident?
 A: The way I think about it, the difference between abelian and non abelian groups is that the former are much better understood. As John Cook says in his answer, abelianness is an extra condition that narrows down the possibilities.
Agreed that abelian groups act as "a receptacle for addition", but additions, or translations, are symmetries too!
A: I don't think this directly answers the question, but I thought the following perspective is nice: Groups and abelian groups are actually group objects in different categories! Namely:


*

*A group is a group object in Set

*An abelian group is a group object in Mon, the category of monoids


This follows from the observation that the inverse map $\iota:G \to G$ is a monoid homomorphism iff $G$ is abelian, plus the following consequence of the Eckmann-Hilton argument alluded to in some other answers (or see the Wikipedia article on group object):


*

*A monoid is a monoid object in Set

*An abelian monoid is a monoid object in Mon
Edit: The rest of what I wrote doesn't hold; see Todd's comments below for the right statement.

One could do the same for Hopf algebras versus commutative Hopf algebras (over a fixed field $k$):


*

*A Hopf algebra is a group object in $k$-Mod

*A commutative Hopf algebra is a group object in $k$-Alg
Since Hopf algebras are the function algebras of quantum groups, while commutative Hopf algebras are the function algebras of groups, I'm tempted to say on a whim that

abelian groups : groups :: groups : quantum groups

i.e. the difference between abelian groups and groups is analogous to the difference between groups and quantum groups.
I don't know enough about quantum groups or Hopf algebras to say whether this is a significant (or even true) analogy. I'd be happy to hear more about it!

A: Here's how I think about it: if the automorphism group of an object is abelian, this means something very strong about the object.  It sort of means you can affect its structure in two different ways, in any order, and they won't affect each other.  This to me hints that maybe the object consists of separate "pieces" that can only be affected independently.
The first example like this which comes to mind is a direct (commutative ring) product of finite fields of different characteristics.  Each field has a cyclic automorphism group, making it very "simple", and the fields can't map into each other, meaning they can only be affected "independently."
So I'd posit that if you want to think of abelian groups as symmetry groups, you can imagine them as the symmetry groups of objects that "break apart" into "simple" pieces which can't interact with each other (intentionally vague, since I don't want to commit to a particular category).
A: Here's sort of an "opposite" question: Is it possible to naturally and intuitively go from the idea of "groups as group presentations/quotients of the free group" to the idea of groups as symmetry groups? Certainly the reason for why abelian groups are counting things arises immediately in the first context... but of course this isn't a particularly nice place to work, since all sorts of things are undecidable.
Oh, right, it's totally possible to go from group presentations to symmetry groups, although not particularly nicely. There is, of course, a standard construction given a group presentation of a guy on which the group acts... the problem is, we either have to work with a silly definition of automorphism which nobody really uses in real life, or introduce some rather gross and unintuitive combinatorial gadgets that complicate things, or accept that there are a bunch of extra automorphisms that the presentation doesn't account for, and which in particular will generally (always?) make the automorphism group non-abelian... Is there a good way to get around this problem?
ETA: Now that I think about it, though, this isn't even necessarily the core issue with the "co-question." It's pretty easy to think about the free abelian group: It's just how we count apples and oranges if we don't want 3 apples and 3 oranges to make 6 fruits. (If you don't want to deal with negative oranges, fine; it's how you, a stock speculator, keep track of your investments [counting short sells] if you -- for good reason -- don't want to mix up your shares of Reynolds Firearms and Roslin Pharmaceuticals.) From there, it's an easy abstraction to think, well, if I lay out my apples and oranges on the table, then the order they're in doesn't matter. And then you (now you're a mathematician and not a grocer) might think, oh, but what if order does matter, which gets you to the free group. But the conceptual leap from there to general finitely presented groups is non-obvious to me. Maybe I should go back to stacking these cantaloupes... 
A: From a representation theory perspective, abelian groups are distinguished by the richness of their invertible representations, since (in the locally compact case), the groups are completely determined by one dimensional complex characters.  For nonabelian groups, the one dimensional characters only determine the abelianization, and one must find actions on larger vector spaces to recover the full group structure.  Abelian groups admit a straightforward Fourier transform, which is a change of basis between functions on a group and functions on the dual (i.e., character) group.  For a nonabelian group, the situation becomes substantially more complicated, involving representations of quantum doubles.
From a homotopy theory perspective, groups can be delooped once, but abelian groups can be delooped infinitely many times.  This is why you can have Eilenberg-MacLane spaces K(G,n) for n > 1 if and only if G is abelian.  Since these spaces represent cohomology with G coefficients, this is why higher degree cohomology only exists for G abelian.
There are intermediate levels of loopiness, corresponding to commutativity up to homotopy plus some intermediate levels of coherence of choices of homotopies.  The quantum double of a group G, for example, lives on the two-fold loop space G/G.  A discrete two-fold loop space H is automatically an abelian group, since \pi2 of the two-fold delooping is H, and higher homotopy groups are abelian (this is one form of the Eckmann-Hilton argument).  As Qiaochu mentioned, the 2-morphisms in a 2-groupoid with one object and one 1-morphism form a natural two-fold "loop set" which therefore is an abelian group.
A: I also like to think of abelian groups as quite different from (general) groups. To me an abelian group should be thought of as being more like a module than a group. Indeed, abelian groups are the same thing as Z-modules and the category of abelian groups is an abelian category (which has many nice properties not shared by the category of groups). The prototypical examples of abelian categories are categories of modules over a ring. In the case of abelian groups the ring is Z (the ring of integers).
A: I think there is something to this idea that abelian groups are only groups "by accident".  Here is at least a construction of the category of commutative monoids which does not mention the monoidal operation directly and cannot be easily modified to yield a definition of the category of all monoids.
Let Fin denote the category of finite sets and all maps, and let Span denote the category of spans in Fin: its objects are the objects of Fin, and a morphism from X to Y is a diagram X ← Z → Y in Fin.  The identity span on X is X ← X → X, and composition of spans is by pullback.  This is properly a 2-category, so make it into a 1-category by replacing each Hom-category by its set of isomorphism classes of objects (discarding the automorphisms).
This category has finite products—they are given by disjoint union of the underlying sets.  Let Comm be the category of finite-product-preserving functors from Span to Set.  Then Comm is equivalent to the category of commutative monoids.
A: It almost seems that the "non" in non-abelian is a positive quality.  
I remember my advisor saying that "nonlinear" isn't a hypothesis but the absence of a hypothesis.  In that sense "nonlinear" means "not necessarily linear."  But you can turn nonlinearity into a positive assertion by looking at things that are nonlinear in a specific way, such as strict convexity, that excludes linearity.  
Along the same lines, maybe the key is to define non-abelian groups in a positive way, to focus on what these groups have that abelian groups lack.  Or to do this for some subset of non-abelian groups.  
A: Here's a totally different suggestion, which is why I'm writing it separately.  As has already been remarked, Ab has the really nice property that it's enriched over itself and monoidal closed.  We don't have this property in Grp because Hom(A, B) doesn't have a natural notion of composition if A != B.  One can, however, construct a category with the elements of Hom(A, B) as objects and invertible functors fixing A and B as morphisms, and this category is a groupoid.  In other words, we should think of Grp as enriched over Gpd - even better, we should think of Grp as living in Gpd, which is enriched over itself and Cartesian closed.  So one reason Grp feels different from Ab is that Grp is neither Cartesian nor monoidal closed.
I don't know whether Gpd feels more similar to Ab than Grp does, though.  
A: A group is a category with a single object and all morphisms invertible; an abelian group is a monoidal category with a single object and all morphisms invertible.
This can be seen via the trick that if a set (in this case the morphisms of the category) has two multiplications that commute with each other (in this case composition of morphisms and tensor product of morphisms) and have the same unit (the unique identity morphism in the category), then they agree and are commutative.
A: Look at Cayley's embedding Theorem which realises every group as a group of permutations. Permutations are functions (from a set to itself . . .). 
So an abelian group  is a set of commuting permutations. For functions, commuting is a serious restriction. And so abelian groups have special properties.
Look at Kronecker-Weber theorem in Algebraic Number Theory. The group of field automorphisms being abelinn means the automorphism functions should commute with each other. So the field has to be very special kind (contained ina field generated by roots of the equation $x^n-1=0$.)
