The answer appears to be in the paper Homotopy groups of symplectic groupsHomotopy groups of symplectic groups by Mimura and Toda. They claim the calculation was already in a paper of HarrisHarris, but that was stated in terms of a symmetric space and it's not immediately obvious to me how to translate into information about the groups.
They state that the group is $\mathbb Z_{(2n+1)!}$ if $n$ is even and $\mathbb Z_{(2n+1)! \cdot 2}$ if $n$ is odd, which agrees with your data.