Consider a set of measured discrete-time signals $y_t \in \mathbb{R}^L$, which are captured from a dynamic physical system with underlying states $x_t \in \mathbb{R}^N$. Let's assume we have more sensors than relevant state variables, i.e., $N < L$, and that we have plenty of time-series data for $y_t$.
My question is: given only the time-series data for $y_t$ (and no other information about the system), are there any established methods for estimating state signals $x_t$ that:
- capture all relevant system dynamics and
- (optional) are physically meaningful/insightful?
One Possible Solution:
So far, what I've come up with is the following. First, we approximate the system as linear and write: \begin{align} x_{t+1} &= A x_t \\ y_t &= C x_t, \end{align} where $A$ is $(N \times N)$ and $C$ is $(L \times N)$, similar to the standard state-space representation of a linear dynamical system with no known inputs. We can then combine this system of equations as follows: \begin{equation} y_{t+1} = C x_{t+1} = C A x_t = C A C^+ y_t, \end{equation} where $(\cdot)^+$ denotes the pseudoinverse. We can then estimate the matrix product, $M = C A C^+$, by constructing two time-shifted $y$ matrices and computing: \begin{equation} Y_2^T = M Y_1^{T-1} \implies Y_2^T \cdot \left(Y_1^{T-1}\right)^+ = M, \end{equation} where, given $j > i$, \begin{equation} Y_i^j = \begin{bmatrix} y_i & y_{i+1} & \cdots & y_{j-1} & y_j \end{bmatrix}. \end{equation} This step appears similar to the approach taken in dynamic mode decomposition. Note that at this point, $M$ is $(L \times L)$, so provided that we have at least $T = L + 1$ time samples of $y_t$, we can compute an overdetermined estimate of $M$.
Then, we can take a truncated singular value decomposition approach, where we write: \begin{equation} M = C A C^+ = U \Sigma V^*. \end{equation} Then, we factor $\Sigma$, which is square and diagonal, into two matrices and take the square root of each, i.e., \begin{equation} M = \left( U \sqrt{\Sigma} \right) \left( \sqrt{\Sigma} V^* \right). \end{equation} We can then truncate the excess columns of the first $\sqrt{\Sigma}$ and the corresponding rows of the second, and write \begin{align} C &= U \sqrt{\Sigma}\\ A C^+ &= \sqrt{\Sigma} V^*, \end{align} where the $\sqrt{\Sigma}$ in the first equation is $(N \times L)$ and that in the second is $(L \times N)$. (The choice to have $A$ in the second equation rather than in the first was arbitrary.)
Thus, $A$ is given by \begin{equation} A = A C^+ C = \sqrt{\Sigma} V^* U \sqrt{\Sigma}. \end{equation}
Now that we've solved for the update matrix $A = \sqrt{\Sigma} V^* U \sqrt{\Sigma}$ and the output matrix $C = U \sqrt{\Sigma}$, we can compute the state signals by \begin{equation} x_t = C^+ y_t. \end{equation}
Assuming the truncated singular values were negligible, it would seem that we've arrived at a representation that captures all relevant system dynamics (my goal #1). But physical insight is rather unlikely, I'd imagine.
A Simpler Solution:
A more straightforward approach I've considered is to simply take the first $N$ principal components of the data matrix $Y_1^T$ (as defined above). We can then call those the state variables, given by \begin{equation} W y_t = x_t, \end{equation} where $W$ is the PCA projection matrix truncated to $N$ components.
Another Alternative Solution:
Another approach I've taken is to use an autoencoder, wherein the hidden variables give a reduced-order estimate of the system dynamics, but again, physical insight is unlikely.
Example of Desired Physical Insight:
My stated goal #2 is admittedly rather vague, so as an example, consider a system of coupled oscillators. In such a setup, the system naturally alternates between two distinct modes of operation, and the system overall usually exists in a superposition state between the two. Ideally, whatever reduced-order state representation results from an analysis of the measured data, I'd like such distinct operating modes to be evident. Perhaps this is not as unlikely as I anticipate, but neither does it seem to be guaranteed by either of the methods I described.
Restating the Question:
So, to reiterate my original question: are there any other, more elegant and/or general approaches to this problem? Ideally, ones that might even yield some physical insight?
If not, is it because what I'm asking is impossible? Are there additional assumptions that I would need to make to make the problem feasible?
