There are all manners of theories [generalizing the notion of derivative][1]. Amongst them is the [fractional calculus][2], a rich theory which gives a sense to the derivation and integration of non-integer (i.e. rational, real, complex) order, that are the real/complex number powers of the differentiation ($D$) and integration ($J$) operators: 

$Df(x)=\frac{d}{dx}f(x)$

$Jf(x)=\int_{0}^{x}f(x)dx$

Along these lines, one may think about *transfinite iteration* of $D$ and $J$ as well. While $D^{(n)}$ and $J^{(n)}$ are quite well-defined and well-behaved operators for the natural number $n$, I haven't found any convenient definition of $D^{(\alpha)}$ or $J^{(\alpha)}$ for the ordinal $\alpha\geq \omega$ in the literature if there is any. 

Due to the similarity between natural and ordinal numbers, it seems the only difficulty is to formulate a definition of the differentiation operator in the *limit* steps like $D^{\omega}$ or $D^{\omega+\omega}$. One straightforward (but not necessarily well-defined, natural or fruitful) way to do so is to think about $D^{\omega}$ as a functional limit of $D^{n}$s in a certain function space. Though, I am not sure if it is the most clever approach. Anyway, it is somehow "natural" to expect that any $D^{\omega}$ operator demonstrates certain properties like: $D^{\omega}x^{n}=0$ for every $n\in \omega$.  

Also, the spaces of smooth and analytic functions, $C^{\infty}$ and $C^{\omega}$, don't seem to capture the essence of the very notion of the *$\omega$-the derivative* of a function, particularly because they don't suggest a clear way of calculating transfinite successor differentiation operators, $D^{\omega+1}$, $D^{\omega+2}$, $D^{\omega+3}$, etc.

> **Question.** Is there any paper in which transfinite derivatives of (real/complex) functions are defined/used? If so, what sort of applications do they have? 

-------------

**Update.** Due to the answer that Andrés mentioned in [his comment][3], it turned out that defining $D^{\omega}$ operator as the limit of $D^{n}$s gives rise to a trivial notion. So maybe a more *direct* approach is needed here.    

  


  [1]: https://en.wikipedia.org/wiki/Generalizations_of_the_derivative
  [2]: https://en.wikipedia.org/wiki/Fractional_calculus
  [3]: https://mathoverflow.net/questions/303660/how-to-define-transfinite-derivatives-of-a-function#comment756892_303660