These two papers answer your question in the introductory paragraphs:
- Norbert Wiener's "The historical background of harmonic analysis"
and
The characteristic-function approach still abounds in generating series related to combinatorics in the umbral calculus / Sheffer sequences / finite operator calculus of Rota et al., where one might define the umbral variables as moments of distributions, defined by characteristic functions, and, of course, in quantum field theory and statistical mechanics with their diverse partition functions and cumulant-moment expansion theorems and associated enumerative diagrammatics, incuding Feynman diagrams, (cf. OEIS A036040 and A127671). E.g., the Laplace transform version gives \begin{gather*} (b_\cdot)^n = b_n = (-1)^n\left.\frac{d^n}{dt^n}\langle\exp(-tx)\rangle\right|_{t=0} \\ = (-1)^n\left.\frac{d^n}{dt^n} \int_0^\infty \exp(-tx)\operatorname{pdf}(x) dx\right|_{t=0} \\ = \int_0^\infty x^n\operatorname{pdf}(x) dx = \langle x^n\rangle, \end{gather*} where the characteristic function for the probability distribution function $\operatorname{pdf}(x)$ is
$$\langle\exp(-tx)\rangle = \int_0^\infty \exp(-tx)\operatorname{pdf}(x) dx .$$
There is an analogous Fourier transform characteristic function
$$\langle\exp(ixt)\rangle.$$
The gaussian distribution and the central limit theorem are key historical focal points in this appoach to probability theory, which is rife with enumerative combinatorics.
More recently, free probability theory employs the Cauchy transform to define characteristic functions for the generating functions of free moments, related to noncrossing partitions, parking functions, random matrices and the Wigner semicircular distribution—the counterpart to the gaussian distribution (cf. "A Simple Introduction to Free Probability Theory and Its Application to Random Matrices" by Xiang-Gen Xia, and A134264.)
(Update 4/1/2021) A walkthrough
First, for the correspondence, see the simple examples in this excerpt ("History of Probability (Part 5) – Laplace (1749-1827)" from ?) of Laplace's use of generating functions (originating with Euler, but a phrase Laplace coined) to encode information on combinatorics ("putting balls in urns" as Rota once put it) and associated probabilities. A more sophisticated example is given by Ulrich in Quora. These generating functions were often generated from difference (recurrence) or differential equations or a mix thereof.
Note that Euler's integral rep for the Euler beta function is in the denominator of the ratio in Problem I of the excerpt and in this case is a convolution of probabilities (some refs state Laplace did not use convolutions). Now read Wiener's survey of the history of harmonic analysis up to the end of the second paragraph of page 59, in which convolution, or Faltung, is stressed. For another perspective on Laplace and probability generating functions, read up to page 551 of Mackey's survey.
Now jump to Loeb's "The world of generating functions and umbral calculus" and read in the section summarizing DR&S:
Thus, Doubilet, Rota, and Stanley have accomplished a tour de force —presenting a unified treatment of all known types of generating functions, and justifying our expectations as to what new types are yet to be found. [Stanley 1986] “The explanatory paradigm based on incidence algebras is this: connected with each special algebraic operator on a ‘variety’ of generating functions is a family of partially ordered sets.. . . The fundamental operation of convolution in the incidence algebra reflect the algebraic operator in questin on generating functions. In this way, the particular algebraic operation acquires a combinatorial interpretation.”
Rota had two careers--first in probability theory and associated operators, second in the umbral / finite operator / Sheffer polynomial calculus and associated combinatorics. Di Nardo nicely summarizes the co-evolution of these two passions of Rota in "Symbolic calculus in mathematical statistics: A review."
(Joseph Kung in "Gian Carlo Rota: A biographical memoir" gives a broader sketch of Rota's interests and contributions, dividing his work into three periods.)