One of the reasons for the ubiquity of the Gaussian is displayed in what is probably the most electrifying half page of scientific prose ever written---Maxwell's argument that the distribution of the velocities of molecules in the ideal gas is Gaussian (now known as the Maxwell-Boltzmann distribution).    The only physical assumptions used are that the density function depends only on the absolute value of the velocity (and not the direction) and that the components in the directions of the coordinate axes are statistically independent.  Mathematically, ths means that the only functions in $3$-space which depend  only on the distance $r$ from the origin and which split as the product of three functions of one variable are those of the form  $ae^{br^2}$.  Maxwell does this by inspection but it is easy to give a rigorous proof (under very weak smoothness conditions) and the result holds, of course, in any dimension greater then or equal to $2$. Maxwell's reasoning can be found in his collected papers, or, more accesssibly, in Hawking's anthology "On the Shoulders of Giants".