3
$\begingroup$

Poincaré plots are used to depict recursions or time series data by interpreting $k$ consecutive values $x[i-(k-1)],\,\dots,\,x[i-1],x[i]$ as a point/vector $\boldsymbol{x}_i:= \left(x_{i,1},\,\dots,\,x_{i,k-1},x_{i,k}\right)^T\in\mathbb{R}^k$

Question:
given $n$ vectors $\boldsymbol{x}_1,\,\dots,\,\boldsymbol{x}_{n-1},\boldsymbol{x}_n$ defined as above,
what information about the qualitative behavior of the series $x[1],x[2],\,\dots,\,x[n+k]$
can be extracted from the eigenvectors and eigenvalues of the covariance matrix $\left(\boldsymbol{x}_1,\,\dots,\,\boldsymbol{x}_{n-1},\boldsymbol{x}_n\right)\left(\boldsymbol{x}_1,\,\dots,\,\boldsymbol{x}_{n-1},\boldsymbol{x}_n\right)^T$

$\endgroup$
3
  • $\begingroup$ Eigenvalues of the covariance matrix are variances of certain linear combinations of the $k$ components of the vector and those linear combinations are uncorrelated with each other. But I'm guessing that's not enough to address what you're asking. $\endgroup$ Commented Aug 16 at 0:50
  • $\begingroup$ @ManfredWeis: Assuming the dynamical system is initialized at the invariant distribution, then $C_{i,i+k} = R(k)$, where $R(k)$ is the $k$-lag covariance matrix. If the process abides by a linear dynamical law $x[n+1] = \rho x[n]+\xi_{n+1}$ (and under the proper assumptions), $R(k) = \rho^k R(0)$. If you "expand" $R(0)$, you get that $R(1)-R(3) = \rho$ -- a critical parametric information for the dynamics. This gets more interesting at the higher-dimensional setting where you can recover the "interacting network" of the dynamical system under the proper assumptions on the dynamics. $\endgroup$ Commented Aug 16 at 13:12
  • 1
    $\begingroup$ All-in-all, the information entailed in $C$ depends on the underlying dynamical law. For reference, I also explored the role of these lags for structure identification at arxiv.org/abs/2312.05974 and at ojs.aaai.org/index.php/AAAI/article/view/26085 $\endgroup$ Commented Aug 16 at 13:16

1 Answer 1

1
$\begingroup$

This is a partial answer, for the record, and hopefully somehow helpful.

Remark 1. The multi-lag empirical covariance matrix $\widehat{C}$ in your question may contain parametric information about the underlying dynamical law generating the time series -- what information will depend on the dynamics itself.

For simplicity, in what follows, I will consider the exact covariance $C = \mathbb[X X^{\top}]$ computed with the expected value, instead of the empirical one $\widehat{C} = \frac{1}{T+1}\sum_{\ell = 0}^{T} \mathbf{x}_{\ell} \mathbf{x}_{\ell}^{\top}$ computed with a finite sample mean.

1D Linear Dynamical System. More concretely, consider a discrete-time linear stochastic dynamical system $$ x[n+1] = \rho x[n] + \xi[n+1] $$ with $\left|\rho\right|<1$; $(\xi[n])_n$ i.i.d. with finite moments and zero mean; and $x[n]$ independent of $\xi[n]$ for all $n$. Assume that the process $(x[n])$ is initialized at the invariant distribution. Then,

$$ C_{i,i+k} = \mathbb{E}\left(x[i]x[i+k]\right) =: R_k = \rho^k R_0, $$

where $R_0 = \sum_{\ell=0}^{\infty} \rho^{2\ell} = 1/(1-\rho^2)$ is the correlation and $C_{i,i+k}$ stands for the $(i,i+k)$ entry of the (limit) covariance matrix $C$, namely, it is the $k$-lag correlation.

These yield two useful estimators for the main parameter $\rho$

$$ \frac{C_{i,i+1}}{R_0} = R_1 (R_0)^{-1} = \rho \,\,\,\,\,\,\,\,\,{\sf (Granger)}$$ $$ C_{i,i+1} - C_{i,i+3} = R_1 - R_3 = \sum_{\ell=0}^{\infty} \rho^{2\ell+1}-\sum_{\ell=0}^{\infty} \rho^{2\ell+3} = \rho, $$

where the latter identity holds from telescopy. Indeed, $\rho$ is a key parameter for the qualitative properties of the linear dynamics.

Multivariate Linear Dynamical System. In this case, the dynamics is given by the counter-part $$ x[n+1] = A x[n] + \xi[n+1] $$ where $A$ is an $N\times N$ symmetric stable matrix whose support entails the graph of interactions among the nodes $m=1, 2, \ldots, N$.

The same estimators extend to this setting $$ C_{i,i+1} (R_0)^{-1}= R_1 (R_0)^{-1} = A \,\,\,\,\,\,\,\,\,{\sf (Granger)}$$ $$ C_{i,i+1} - C_{i,i+3} = R_1 - R_3 = A, \,\,\,\,\,\,\,\,\, (\star \star)$$ where now $C_{i,i+k} = R_k$ is a matrix: The $k$-lag covariance matrix. The sample mean study of the relation $(\star \star)$ can be found at [1].

Unveiling the graph of interactions. Assuming that $A$ is nonnegative: Nodes $i$ and $j$ are connected if $A_{ij}>0$, otherwise, if $A_{ij}=0$, then they are disconnected. With this in mind and in view of the identity $(\star \star)$, we have that the set of vectors $\left\{F_{ij}\,:\,i,j = 1,2,\ldots,N\right\}$ defined as

$$F_{ij} \overset{\Delta}= \left(\left[R_1\right]_{ij},\left[R_2\right]_{ij},\ldots,\left[R_M\right]_{ij}\right)$$

for $M>3$ is consistently linearly separable, i.e., there exists a hyperplane that separates the vectors associated with connected pairs $\left\{F_{ij}\,:\,i\sim j\right\}$ from the vectors of disconnected pairs $\left\{F_{ij}\,:\,i\nsim j\right\}$. This means that $C$ entails information about the underlying graph of interactions: To recover, you need to properly cluster the set $\left\{F_{ij}\right\}$. The technical development of this discussion can be found at [2] and [3].

Remark 2. The parametric and structural information entailed in $C$ and discussed rely on the linear dynamics underlying the time-series. For other dynamical laws, the guarantees should be reworked accordingly.

[1] Chen et al., An Unbiased Symmetric Matrix Estimator for Topology Inference under Partial Observability, IEEE Signal Processing Letters, 29(02): 1257-1261, 2022.

[2] Machado et al., Recovering the Graph Underlying Networked Dynamical Systems under Partial Observability: A Deep Learning Approach, Proceedings of the 37th AAAI Conference on Artificial Intelligence, Washington D. C., 9038-9046, Feb. 2023.

[3] Santos et al., Learning the Causal Structure of Networked Dynamical Systems under Latent Nodes and Structured Noise, Proceedings of the 38th AAAI Conference on Artificial Intelligence, Vancouver, Canada, Feb. 2024.

P.S.-- My apologies: You asked something (how the spectra of the empirical covariance $\widehat{C}$ universally informs about the qualitative properties of the system) and I answered something else (how the exact $C$ itself can inform about certain built-ins of a linear dynamical law). Perhaps someone can provide a more complete answer.

$\endgroup$
1
  • 1
    $\begingroup$ even if your answer is not what I asked for it is still valuable information related to the topic $\endgroup$ Commented Aug 17 at 11:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .