I'm shocked that noone has mentioned the Quaternion groupQuaternion group! This thing is a counterexample to lots of basic questions you'd come up with while learning (finite) group theory.
For example (although not really a counterexample to a specific question), if you know the semidirect product construction and Sylow theorems and are trying to classify groups of low order, the quaternion group is the first group you can't construct as a semidirect product of cyclic groups. This can be an entry point for the extension problem for groups and cohomology of groups.