In Science and Hypothesis, chapter XI, The calculus of probabilities, Henri Poincaré deals informally with the fundamental problem of interpolation. He concludes (see http://ia600308.us.archive.org/21/items/scienceandhypoth00poinuoft/scienceandhypoth00poinuoft.pdf, page 206):
Why, then, do I draw a curve without sinuosities? Because I consider a priori a law represented by a continuous function (or function the derivatives of which to a higher order are small), as more probable than a law not satisfying those conditions. But for this conviction the problem would have no meaning; interpolation would be impossible; no law could be deduced from a finite number of observations; science would cease to exist.
To my knowledge, Poincaré interpolation was finally addressed in 1992 by Bretthorst using Jaynes' Principle of Maximum Entropy. Please refer to Bayesian interpolation and deconvolution, http://bayes.wustl.edu/glb/deconvolution.pdf. The principle is quite simple:
- By Bayes' rule, we just need to assign the prior probability distribution for the smooth theoretical signal $u(t)$ to be estimated (eq. 10).
- Approximate for instance $\displaystyle \frac{\textrm{d}^{2}u}{\textrm{d}t^{2}}$ numerically by finite differences (eq. 16). We seek for our prior distribution among all continuous distributions having same Euclidean norm $\displaystyle\left\|\frac{\textrm{d}^{2}u}{\textrm{d}t^{2}}\right\| $ (eq. 17).
- Call the Principle of Maximum Entropy: finally, you get a multivariate Gaussian prior distribution (eq. 28) involving the fractional variance regularizer $\epsilon^{2}$.
- Assign a prior distribution for $\epsilon$ and compute its marginal posterior distribution (eq. 61). Then you get for instance its MAP estimate by maximization (fig. 3);
- Estimate $u_{j}$ by maximizing the marginal posterior $p(u_{j}|\epsilon,D,I)$ (eq. 52, fig. 4).
Bretthorst considers that $u(t)$ is to be estimated at $\nu+2$ points where $\nu=\beta(N-1)+1$ (eq. 5), $N$ is the number of observations and $\beta$ is a positive integer. We can think for instance about a regular grid with step $\Delta t/\beta$. In order to make the finite differences (eq. 16) and consequently our estimates more accurate, we should certainly increase $\beta$ even if we want to estimate $u(t)$ at, say, only the $N$ measurement points. In practice, we actually observe that the interpolation converges quickly as $\beta$ increases. Therefore the following problems may arise:
Problem 1: Can we pass to the limit $\beta\to+\infty$ in the interpolation problem?
$p(u_{j}|\epsilon,D,I)$ (eq. 52) involves the eigenvalues and the eigenvectors of the $j^{th}$ cofactor of the square matrix $g_{ik}=\epsilon^{2}R_{ik} + S_{ik}$ (eq. 43-52) of dimension $\nu+2$. Therefore, there are two possibilities:
- Either we can get those eigenvalues and eigenvectors analytically for any $\beta$ and $\epsilon$ or we can approximate them. Then, we would just need to take the limit in the series $h_{l}(u_{j})$ (eq. 43).
- Or we can get those eigenvalues and eigenvectors only for the infinite matrix $g_{ik}$. Then, we may need to justify that we can interchange the limits.
Problem 2: Can we pass to the limit $\beta\to+\infty$ in the deconvolution problem?
Now the matrix $S_{ik}$ depends on the impulse response function $r(t)$ (eq. 112) so that we can't use the nice structure of the interpolation matrix (eq. 35).
