Skip to main content
1 of 2

Order statistic of Markov chain sample path and related probabilities

Consider a 1D sample path, denoted as $\{X(1), ..., X(t), ..., X(n)\}$, generated from a discrete time finite state (time homogeneous) Markov chain over states $\{1,...,m\}$, with transition probability matrix $P$, and initial occupation probability vector $p_{0}$. Let $X_{(1)} \leq ... X_{(j)} \leq ... \leq X_{(n)}$ denote the non-descending rearrangement of the sample path. As is well known, $X_{(j)}$ is called the $j$-th order statistic for this sample path of length $n$.

Problem 1. For $i,j\in\{1,...,m\}$, I want to compute the probability mass function (pmf) for $X_{(j)}$, that is, $\mathbb{P}\left(X_{(j)} = i\right)$. This probability for the easier i.i.d. case is well known, see for example p.23, left column top, here. How to do this for the Markov case?

Problem 2. Compute $\mathbb{P}(X(t) = X_{(j)})$.

Using total probability and Bayes rule, the probability for Problem 2 is $\sum_{i=1}^{m}\mathbb{P}(X_{(j)} = i \:|\:X(t) = i) \: \mathbb{P}(X(t)=i)$. The second term in the product can easily be written in terms of $p_{0}, P$ and $t$. But I don't know how to compute the conditional (I tried to think in terms of taboo probabilities with not much luck). Or perhaps there is a better strategy altogether?

Any reference or ideas are welcome. Assume irreducible, aperiodic if that helps.