This is a cross-posting where I couldn't get an answer. In the meantime I have tried to improve the original logic:
As in Takens original paper about his embedding theorem, consider a compact $m$-dimensional manifold $M$ and a smooth map $f:x_t \in M\rightarrow x_{t+1}\in M$ that expresses the state-evolution in a discrete-time state-space system. Consider a second. continuous map $\phi:x_t \in M \rightarrow y_t\in \mathbb{R}^k$ where $y_t$ is the vector of actually observed variables at time $t$ and for this particular example $k=1$.
According to Takens (1981) there exists an embedding of $M$ via $\Phi_{f,\phi}:M \rightarrow \mathbb{R}^{2m+1}$ with $\Phi_{f,\phi}(x)=(\phi(x),\phi(f(x)),...,\phi(f^{2m}(x))^T$.
(Explicitly, given a sufficient amount of observations $y_t$, the embedding is $Y_t=(y_t, y_{t-\tau},y_{t-2\tau},...,y_{t-m\tau})^T$ where $\tau$ and $m$ have to be determined from the available data.)
Now comes the part where I try to justify the use of e.g. a universal approximator like Feedforward Neural Networks to forecast $y_{t+1}$ from $Y_t$:
Since an embedding is an injective, continuous map and $f$ is smooth there exists a continuous map via composition:
$$\tilde{\Phi}_{f,\phi}=\Phi_{f,\phi}\circ f\circ\Phi^{-1}_{f,\phi}$$ that maps as $\tilde{\Phi}_{f,\phi}:Y_t\rightarrow Y_{t+1}$.
Since $\phi$ is continuous there exists another continuous map via composition:
$$\tilde{\phi}=\phi\circ\Phi^{-1}_{f,\phi}$$ that maps as $\tilde{\phi}:Y_t\rightarrow y_t$.
Then there exists a third continuous function via composition:
$$\Phi^*=\tilde{\phi}\circ\tilde{\Phi}_{f,\phi}$$
that maps as $\Phi^*:Y_t\rightarrow y_{t+1}$. That means that knowledge of $\Phi^*$ allows to use the embedding vector $Y_t$ to forecast the future realization of the time-series $y_{t+1}$. However, $\Phi^*$ is unknown in most cases so it has to be approximated.
Since $M$ is compact and every map considered so far is continuous, we are only dealing with compact sets, especially $Y_t \subset_{compact}\mathbb{R}^{2m+1}$. The universal approximation theorem for Neural Networks states, that every continuous function on a compact subset of $\mathbb{R}^n$ can be approximated by a one layered-feedforward network $f_{NN}$. Since $\Phi^*$ is a continuous function on $Y_t$ which only evolves in such a compact set, $\Phi^*$ can be approximated by $f_{NN}$.
My question is: Is this derivation correct? I want to justify my approach in my Master's thesis but haven't found anything comparable in the literature. If you know a book/paper that deals with this problem, I would highly appreciate recommendations as well.
