It seems to me that a significant proportion of work in probability theory, statistics and machine learning are on understanding continuous-valued, relatively weakly dependent, or linear dependent stochastic processes (correlation, random matrices, random fields, various CLT assuming weak/specific/limited dependency; or Dynamic Time Warping alg for aligning similar random curves).
- I'm wondering if similar framework/method has been (partially) developed for discrete valued stochastic processes
- regression problems when $X_t$ and $Y$ are of different geometric dimension (one random curve the other random variable)
More specifically,
Question_1: Is there a probabilistic/statistical measure of dependency for discrete-valued random vectors/stochastic processes? Is there a natural geometry associated? What kind of dependencies are already considered/quantified?
I believe correlation and copula probably may fail, due to violation of vector space axioms (if using correlation) and continuity assumption on the marginal distributions (if using copula).
Question_2: What alignment methods / clustering algorithms for discrete valued stochastic processes are considered?
I guess one way to understand this is to develop a suitable metric/measure/topological structure on the space of discrete valued stochastic processes (along with its sample counterpart). This is perhaps related to Question_1. While in Question_1 the emphasis is how $Y_t$ depends on $Y_{t-1}$ and/or $X_t$, etc., here we also search for similarities from local to global scales. Does $X_t$ look similar to parts of itself? Do ${X_t}$ and $Y_t$ look alike?
Question_3: Consider the statistical inference problems on such probabilistic phenomena. That is, given data on more than two stochastic processes (at least one is discrete valued), how do we solve the inverse problem of choosing a set of good probabilistic models to describe them jointly? What kind of spaces of functions do we choose to work on, what kind of metric/measure (related to the above)?
This is perhaps related to Hidden Markov Model and recurrent neural network (RNN). However, the former can be a bit too simplistic and the latter may be difficult and time-consuming to train. Maybe some random field models are out there already. I would be happy to hear about them.
Question_4: Let $\{X_t\}$, $\{Y_s\}$ be time series, sampled at different rates. For example, $X_t$ is collected weekly, and $Y_s$ is collected hourly or even every minute (maybe best thought of as a random curve).
If we want to predict $\{X_t\}$ based on $\{Y_s\}$, what kind of methods/models/spaces to consider? Things can become worse if $\{Y_s\}$ is multivariate random vectors/curves sampled at different rates themselves. Is there a framework on regressing random curves/surfaces on random variables?
