No, your impression is not correct. -- A counterexample is the group ${\rm S}_5$, whose elements have orders $1, 2, 3, 4, 5$ and $6$.
As to groups with spectrum (i.e. set of element orders) equal to $\{1, \dots, n\}$: the maximum here is 8, and for $n = 8$ the only such group is an extension of ${\rm PSL}(3,4)$ by a unitary automorphism -- see herehere.