Because I still have no idea how it is possible for me to write down seemingly important equations ... that don't make any sense (at least for me) and because I haven't got any helpful comment so far, I'll be happy to offer a +100 bounty (that is, almost all my reputation!) not for a definitive anwser to this weird question but only for any serious, relevant feedback, thought, opinion or advice.

In many signal processing calculations, the prior probability distribution of the theoretical signal of interest (not the noisy experimental signal) is required. Here is the concrete problem from which my question has arisen:

Is there a Bayesian theory of deterministic signal? Prequel and motivation for my previous question

In random signal theory (à la Shannon), this probability distribution typically is a stochastic process, e.g. an i.i.d. stochastic process if you are a frequentist or an exchangeable stochastic process if you are a Bayesian, in the most basic cases.

What do such probability distributions become in deterministic signal theory/dynamical system theory?, that is the question.

To make it simple, consider a discrete-time real deterministic signal

$ s\left( {1} \right),s\left( {2} \right),...,s\left( {M} \right) $

For instance, it can be obtained by sampling a continuous-time real deterministic signal.

By the standard definition of a discrete-time deterministic dynamical system, there exists:

a phase space $\Gamma$, e.g. $\Gamma \subset \mathbb{R} {^d}$, $\Gamma = \left[ {0,1} \right]$, etc.

a state-space equation $f:\Gamma \to \Gamma $ such as $z\left( {m + 1} \right) = f\left[ {z\left( m \right)} \right]$;

an output or observation equation $g:\Gamma \to \mathbb{R}$ such as $s\left( m \right) = g\left[ {z\left( m \right)} \right]$;

an initial condition $ z\left( 1 \right)\in \Gamma $ in the domain of definition of $f$.

Hence, by definition we have

$\left[ {s\left( 1 \right),s\left( 2 \right),...,s\left( M \right)} \right] = \left\{ {g\left[ {z\left( 1 \right)} \right],g\left[ {f\left( {z\left( 1 \right)} \right)} \right],...,g\left[ {{f^{M - 1}}\left( {z\left( 1 \right)} \right)} \right]} \right\}$

or, in probabilistic notations

$p\left[ {\left. {s\left( 1 \right),s\left( 2 \right),...,s\left( M \right)} \right|z\left( 1 \right),f,g,\Gamma ,d} \right] = \prod\limits_{m = 1}^M {\delta \left\{ {g\left[ {{f^{m - 1}}\left( {z\left( 1 \right)} \right)} \right] - s\left( m \right)} \right\}} $

Therefore, by "total probability and the product rule", the "marginal joint prior probability distribution" for a discrete-time deterministic signal conditional on phase space $\Gamma$ and its dimension $d$ formally/symbolically writes

$p\left[ {\left. {s\left( 1 \right),s\left( 2 \right),...,s\left( M \right)} \right|\Gamma ,d} \right] = \int\limits_{{\mathbb{R}^\Gamma }} {{\text{D}}g\int\limits_{{\Gamma ^\Gamma }} {{\text{D}}f\int\limits_\Gamma {{{\text{d}}^d}z\left( 1 \right)\prod\limits_{m = 1}^M {\delta \left\{ {g\left[ {{f^{m - 1}}\left( {z\left( 1 \right)} \right)} \right] - s\left( m \right)} \right\}p\left( {z\left( 1 \right),f,g} \right)} } } } $

Should phase space $\Gamma$ and its dimension $d$ be also unknown *a priori*, they should be marginalized as well so that the most general "marginal prior probability distribution" for a discrete-time deterministic signal I'm considering formally/symbolically writes

$p\left[ {s\left( 1 \right),s\left( 2 \right),...,s\left( M \right)} \right] = \sum\limits_{d = 1}^{ + \infty } {\int\limits_{\wp \left( {{\mathbb{R}^d}} \right)} {{\text{D}}\Gamma \int\limits_{{\mathbb{R}^\Gamma }} {{\text{D}}g\int\limits_{{\Gamma ^\Gamma }} {{\text{D}}f\int\limits_\Gamma {{{\text{d}}^d}z\left( 1 \right)\prod\limits_{m = 1}^M {\delta \left\{ {g\left[ {{f^{m - 1}}\left( {z\left( 1 \right)} \right)} \right] - s\left( m \right)} \right\}p\left( {z\left( 1 \right),f,g,\Gamma ,d} \right)} } } } } } $

where ${\wp \left( {{\mathbb{R}^d}} \right)}$ stands for the powerset of ${{\mathbb{R}^d}}$.

Dirac's $\delta$ distributions are certainly welcome to "digest" those very high dimensional "integrals". However, we may also be interested in "probability distributions" like

$p\left[ {s\left( 1 \right),s\left( 2 \right),...,s\left( M \right)} \right] \propto \sum\limits_{d = 1}^{ + \infty } {\int\limits_{\wp \left( {{\mathbb{R}^d}} \right)} {{\text{D}}\Gamma \int\limits_{{\mathbb{R}^\Gamma }} {{\text{D}}g\int\limits_{{\Gamma ^\Gamma }} {{\text{D}}f\int\limits_\Gamma {{{\text{d}}^d}z\left( 1 \right)\int\limits_{{\mathbb{R}^ + }} {{\text{d}}\sigma {\sigma ^{ - M}}{e^{ - \sum\limits_{m = 1}^M {\frac{{{{\left\{ {g\left[ {{f^{m - 1}}\left( {z\left( 1 \right)} \right)} \right] - s\left( m \right)} \right\}}^2}}}{{2{\sigma ^2}}}} }}p\left( {\sigma ,z\left( 1 \right),f,g,\Gamma ,d} \right)} } } } } } $

Please, what can you say about those important "probability distributions" beyond the fact that they should better not be invariant by permutation of the time points, i.e. not finitely De Finetti-exchangeable, otherwise the chronological order, that is the time would be lost (conjecture)?

What can you say about such strange looking "functional integrals" (for the state-space and output equations $f$ and $g$) and even "set-theoretic integrals" (for phase space $\Gamma$) over sets having cardinal at least ${\beth_2}$? Are they already well-known in some branch of mathematics I do not know yet or are they only abstract nonsense?

Clearly, the noninformative case is the most important one. Hence, a definitive answer to my question could be something like this:

If $p\left( {z\left( 1 \right),f,g} \right)$ is the "improper non-informative prior probability distributions" over $\Gamma \times {\Gamma ^\Gamma } \times {\mathbb{R}^\Gamma }$ $p\left( {z\left( 1 \right),f,g} \right) \propto 1$ then the "marginal probability distribution"

$p\left[ {\left. {s\left( 1 \right),s\left( 2 \right),...,s\left( M \right)} \right|\Gamma ,d} \right] = \int\limits_{{\mathbb{R}^\Gamma }} {{\text{D}}g\int\limits_{{\Gamma ^\Gamma }} {{\text{D}}f\int\limits_\Gamma {{{\text{d}}^d}z\left( 1 \right)\prod\limits_{m = 1}^M {\delta \left\{ {g\left[ {{f^{m - 1}}\left( {z\left( 1 \right)} \right)} \right] - s\left( m \right)} \right\}} } } } $

is the improper uniform probability distribution over ${\mathbb{R}^M}$.

Your are free to restrict the sets of the state-space equations and output equations if necessary. However, contrary to what is suggested below, for a discrete-time dynamical system the state space equation $f$ needs not be continuous (the classical counterexample for phase space $\Gamma = \left[ {0,1} \right]$ is the Bernoulli shift that is discontinuous at $1/2$). Hence, *a priori* ${\beth_2}$ of them must be marginalized out.

Those beasts look very important to me because there are many problems of interest where we know *a priori* that the signal is deterministic but we don't known the underlying dynamical system (and output equation) and there is little hope to ever know it (generally speaking, dynamical systems identification is very delicate and difficult). Hence the deterministic model is unknown and not operationnal and by definition it is not legitimate to introduce a (objectively) stochastic one. So how to handle and process those signals properly? In theory, we perfectly know how to do it: just marginalize over all possible dynamical systems. But that does not make sense mathematically, so that it seems that we don't know how to model and process deterministic signals of unknown origin.

Thanks.

14more comments