Let $X$ be a transient Markov chain with countable state space $S(X)$. Let $Y$ be a positive recurrent Markov chain with countable state space $S(Y)$. (Time is discrete.)

Let $A \subseteq S(X)$ be such that, with probability 1, $X$ visits $A$ infinitely many times. In fact, if it helps, let $A$ be such that, with probability 1, there exists a sequence $T_n \to \infty$ such that $\frac{1}{T_n}$(number of visits to $A$ during $[0,T_n]$) $\to a > 0$. (Obviously such sets $A$ exist, e.g. $A = S(X)$.)

1) Is it true that, in fact, $\lim_{T \to \infty} \frac{1}{T}$ (number of visits to $A$ during $[0,T]$) $= a$?

2) Let $y \in S(y)$ be any state; recall that $Y$ was positive recurrent. Is it true that the product chain $(X,Y)$ will visit $(A,y)$ with probability 1? Is there an estimate on $\frac{1}{T}$(number of visits to $(A,x)$ during $[0,T]$), at least for some sequence of $T$?


The answer to 1. is negative. Consider the transient Markov chain $X_n=X_0+n$ on $\mathbb Z$ and choose for $A$ any subset of $\mathbb Z_+$ with different inferior and superior densities.

As regards 2., I assume you want $X$ and $Y$ to be independent. Then the answer to the first question is negative. Same $X$ as before, $X_0$ deterministic, $Y_n=(-1)^n$, $y=(-1)^{X_0}$ and $A=2\mathbb Z+1$. Then $(X,Y)$ never visits $A\times\{y\}$.

  • $\begingroup$ Agree with 1., thanks. 2. What if I add that Y is aperiodic? (And, of course, independent of X.) $\endgroup$ Sep 17 '11 at 16:36
  • $\begingroup$ If Y is aperiodic than $P(Y_n=y) \to \pi_y$, so for any increasing sequence of times $t_n$, the probability of $Y_{t_n}=y$ for some (infinitely many) $n$'s is 1. $\endgroup$ Sep 17 '11 at 19:22
  • $\begingroup$ I think you mean for $t_n$ to be independent of $Y$? In that case, yes, I think an application of Levy's extension of the Borel-Cantelli lemma (for weakly dependent random variables) should do it. Or is there a more obvious reason why this is true? $\endgroup$ Sep 18 '11 at 0:43
  • $\begingroup$ I meant for any fixed sequence $t_n$. It then follows for any random sequence which is independent of $Y$. As for an obvious way to see it: this fact is true also conditioned on any finite history, so you can always choose $n$ large enough so that $\mathbb{P}(Y_{t_n}=y)\ge \pi_y /2$ conditioned on what you observed so far. Thus, you have infinitely many "trials", each having at least some fized probability, given all the previous ones. $\endgroup$ Sep 18 '11 at 18:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.