2
$\begingroup$

For an ergodic Markov Chain $$ \frac{1}{N}\sum_{i=1}^n f(X_i) \rightarrow E_\pi[f] $$ where $\pi$ is the invariant distribution. I am also dealing with a Markovian process (a state space model to be specific) and I have a quantity like the following: $$ \frac{1}{T} \sum_{t=1}^T \log p(x_t \mid x_{t-1},\theta) $$ where the state space model that generated the data is $x(0) \sim p(x_0)$ and the transition model is $x_t \sim p(x_t \mid x_{t-1},\theta)$. Can I apply the ergodic theory in this setting? If so, what would the above sum converge to?

In general, instead of $\frac{1}{T}\sum_{t=1}^T f(X_t)$ what happens if I have $\frac{1}{T}\sum_{t=1}^T f(X_{t-L},\dots,X_t)$?

p.s. I asked this question in cross-validated but no answers or attention up till this point.

$\endgroup$
11
  • 1
    $\begingroup$ I am not sure if I understand exactly your question. However, it appears to me that in your case you can consider the invariant distribution $\tilde{\pi}$ of the random vectors $(X_{t},\ldots,X_{t+L}).$ Then by the Ergodic theorem the limit you are asking should be $\mathbb E_{\tilde{\pi}}(f).$ $\endgroup$
    – user39115
    May 15, 2016 at 15:40
  • 1
    $\begingroup$ What's $\theta$? Your expression looks like it's converging to the entropy rate of the Markov chain. $\endgroup$ May 15, 2016 at 15:46
  • $\begingroup$ $\frac{1}{T}\sum_{t=1}^T f(X_{t-L},\dots,X_t)$ will converge a.s. to $E_\pi[f(X_0,\ldots,X_L)]$. $\endgroup$ May 15, 2016 at 18:12
  • $\begingroup$ @AryehKontorovich in my application $\theta$ is unknown but for the sake of simplicity let's assume $\theta$ is a known constant. I can see the entropy connection due to p.logp term but can you give me the formal definition or a link for this term? $\endgroup$
    – jkt
    May 15, 2016 at 18:30
  • $\begingroup$ @JohnDawkins, well $\pi$ is the invariant distribution for $X_t$ as $t\rightarrow \infty$ right? So how do you compute the expectation $E_{\pi}[f(X_0,\dots,X_L)]$ you mentioned? Don't I need some sort of a joint $\pi$? $\endgroup$
    – jkt
    May 15, 2016 at 18:32

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.