I thought about this issue some time ago and I reached an interesting point of view I think. I believe it is worth to share it here. Of course it may be naive. Please feel free to comment.
Integration (antiderivation if you like) is not a "mapping" between a function and another function of the same dimensionality. After each integration a new "constant" appears. This new constant is a new degree of freedom or dimension.
From the recursive point of view this is very important I believe. Sometimes when you integrate (e.g. polynomials and exponentials/sin/cos) the new constant falls into the same family of functions that contained the original functions. For example, a constant is just a polynomial of degree zero or an exponential with zero "growing rate/frequency" (ie exp(0.x) = const. and cos(0.x)=1).
Notice also that the family of polynomials and exponentials are preserved by summation and multiplication. Polynomials also by function composition. So even a priory complicated expressions are just "solved" using the same procedure by recursion.
Now, when because of the new constant you fall outside of this family, for instance: exp(exp(x) + cte) is not in the same family as exp(x), then you need to grow your family and so your recursive method is more complicated. It may happen (in principle) that the (extended) family is infinite and so you cannot find a method at all! You just need a bible :).
In my view the new constants allow us to put more and more information about the real world into the expression (initial conditions, boundary conditions, some physical parameters, etc). Some times this constants are meaningless, other or do not have physical consequences. Other times they are physically important because your physical system increased in complexity due to this new constant. The new constant expanded the family of functions! More over, from a statistical point of view, sometimes a large number of integration constants are just equivalent to noise. In this case the microscopic description may be impractical but a macroscopic statistical description may be obtained. TheFurthermore, the aggregate contribution of the constants to some "macroscopic/statistical" quantities is incoherent and so the particular values of the constants are washed out.
In other cases, where complex systems as living systems reside, the integration constants are very important and cannot be ignored or "statistically reduced" to a small number. Using the words of Philip W Anderson more is different in this sense. These new integration constants sum up coherently and complexity arises.
Best Regards, Juan I. Perotti