What reasons are there for describing the harmonic oscillator as being so important in physics?
The harmonic oscillator tends to show up when you're expanding a potential function around non-degenerate critical points.
The simplest example is a physical system described by a map $t \mapsto \phi(t) \in \mathbb{R}$. If the energy function for this system has the form $E(\phi) = \frac{1}{2}|\dot{\phi}|^2 + V(\phi)$, with $V$ bounded below, then the lowest energy states are going to be of the form $\phi_0(t) = \phi_0$, where the constant $\phi_0$ is a minimum of $V$, hence a critical point. So, if your map $\phi$ never deviates too much from $\phi_0$ and $\phi_0$ is a non-degenerage critical point, you can approximate the energy function by $E(\phi) = |\frac{1}{2}\dot{\phi}|^2 + \frac{1}{2}V''(\phi_0)(\phi-\phi_0)^2$.
In other words, the harmonic oscillator potential describes small disturbances around "generic" minima of an energy function. This situation comes up all the time in physics. For example: it shows up in Witten's Supersymmetry & Morse Theory paper, which I think would have been well-known to people working on topology and analysis in the 1980s.