*What reasons are there for describing the harmonic oscillator as being so important in physics?*

The harmonic oscillator tends to show up when you're expanding a potential function around non-degenerate critical points.  

The simplest example is a physical system described by a map $t \mapsto \phi(t) \in \mathbb{R}$.  If the energy function for this system has the form $E(\phi) = \frac{1}{2}|\dot{\phi}|^2 + V(\phi)$, with $V$ bounded below, then the lowest energy states are going to be of the form $\phi_0(t) = \phi_0$, where  the constant $\phi_0$ is a minimum of $V$, hence a critical point.  So, if your map $\phi$ never deviates too much from $\phi_0$ and $\phi_0$ is a non-degenerage critical point, you can approximate the energy function by $E(\phi) = |\frac{1}{2}\dot{\phi}|^2 + V(\phi_0)+\frac{1}{2}V''(\phi_0)(\phi-\phi_0)^2$.  

In other words, the harmonic oscillator potential describes small disturbances around "generic" minima of an energy function.  This situation comes up all the time in physics.  For example:  it shows up in Witten's Supersymmetry & Morse Theory paper, which I think would have been well-known to people working on topology and analysis in the 1980s.