In Penrose's book (The Road to Reality, chapter 21) he gives an example of Oliver Heaviside's observation that you can treat differential operators like numbers:
The differential equation $(1+D^2)y = x^5$ can be solved by dividing by $(1+D^2)$ then taking the power series expansion: $$y = (1-D^2+D^4-D^6+\cdots)x^5$$ which evaluates to $y = x^5 - 20x^3 + 120x$.
Apparently this can be made perfectly rigorous!
How is this done? and do you know where I can read more details about this idea?
This is just a fact from linear algebra: if $T$ is a nilpotent transformation of a vector space $V$, then $(1-T)^{-1} = 1 + T + T^2 + \dots$. More generally, the same is true in any commutative Banach algebra (such as the endomorphism ring of a normed complex vector space) if $T$ is of norm less than 1.
In your case, the differential operator $D$ is a nilpotent operator on the vector space of polynomials, and consequently this formula applies.
As people have pointed out in the comments, you may want to look at the Fourier transform, which realizes differentiation as multiplication, which allows you to treat differentiation kind of like a number. (More precisely, there is an isomorphism $F:L^2 \to L^2$ such that $F D F^{-1}$ is equal to multiplication by $x$ on sufficiently nice (e.g. Schwarz) functions.)
In higher dimensions, for instance, this means that if $\Delta$ is the Laplacian, then multiplication by $(1 + |x|^2)^k$ corresponds to applying $(I-\Delta)^k$, and it is thus possible to define an operator $(I - \Delta)^r$ for any real $r$.
