Skip to main content
add tag;
Source Link
Henry.L
  • 8.1k
  • 8
  • 48
  • 74

Consider a 1Done dimensional sample path, denoted as $\{X(1), ..., X(t), ..., X(n)\}$, generated from a discrete time finite state (time homogeneous) Markov chain over states $\{1,...,m\}$, with transition probability matrix $P$, and initial occupation probability vector $p_{0}$. Let $X_{(1)} \leq ... X_{(j)} \leq ... \leq X_{(n)}$ denote the non-descending rearrangement of the sample path. As is well known, $X_{(j)}$ is called the $j$-th order statistic for this sample path of length $n$.

Problem 1. For $i,j\in\{1,...,m\}$, I want to compute the probability mass function (pmf) for $X_{(j)}$, that is, $\mathbb{P}\left(X_{(j)} = i\right)$. This probability for the easier i.i.d. case is well known, see for example p.23, left column top, here. How to do this for the Markov case?

Problem 2. Compute $\mathbb{P}(X(t) = X_{(j)})$.

Using total probability and Bayes rule, the probability for Problem 2 is $\sum_{i=1}^{m}\mathbb{P}(X_{(j)} = i \:|\:X(t) = i) \: \mathbb{P}(X(t)=i)$. The second term in the product can easily be written in terms of $p_{0}, P$ and $t$. But I don't know how to compute the conditional (I tried to think in terms of taboo probabilities with not much luck). Or perhaps there is a better strategy altogether?

Any reference or ideas are welcome. Assume irreducible, aperiodic if that helps.

Consider a 1D sample path, denoted as $\{X(1), ..., X(t), ..., X(n)\}$, generated from a discrete time finite state (time homogeneous) Markov chain over states $\{1,...,m\}$, with transition probability matrix $P$, and initial occupation probability vector $p_{0}$. Let $X_{(1)} \leq ... X_{(j)} \leq ... \leq X_{(n)}$ denote the non-descending rearrangement of the sample path. As is well known, $X_{(j)}$ is called the $j$-th order statistic for this sample path of length $n$.

Problem 1. For $i,j\in\{1,...,m\}$, I want to compute the probability mass function (pmf) for $X_{(j)}$, that is, $\mathbb{P}\left(X_{(j)} = i\right)$. This probability for the easier i.i.d. case is well known, see for example p.23, left column top, here. How to do this for the Markov case?

Problem 2. Compute $\mathbb{P}(X(t) = X_{(j)})$.

Using total probability and Bayes rule, the probability for Problem 2 is $\sum_{i=1}^{m}\mathbb{P}(X_{(j)} = i \:|\:X(t) = i) \: \mathbb{P}(X(t)=i)$. The second term in the product can easily be written in terms of $p_{0}, P$ and $t$. But I don't know how to compute the conditional (I tried to think in terms of taboo probabilities with not much luck). Or perhaps there is a better strategy altogether?

Any reference or ideas are welcome. Assume irreducible, aperiodic if that helps.

Consider a one dimensional sample path, denoted as $\{X(1), ..., X(t), ..., X(n)\}$, generated from a discrete time finite state (time homogeneous) Markov chain over states $\{1,...,m\}$, with transition probability matrix $P$, and initial occupation probability vector $p_{0}$. Let $X_{(1)} \leq ... X_{(j)} \leq ... \leq X_{(n)}$ denote the non-descending rearrangement of the sample path. As is well known, $X_{(j)}$ is called the $j$-th order statistic for this sample path of length $n$.

Problem 1. For $i,j\in\{1,...,m\}$, I want to compute the probability mass function (pmf) for $X_{(j)}$, that is, $\mathbb{P}\left(X_{(j)} = i\right)$. This probability for the easier i.i.d. case is well known, see for example p.23, left column top, here. How to do this for the Markov case?

Problem 2. Compute $\mathbb{P}(X(t) = X_{(j)})$.

Using total probability and Bayes rule, the probability for Problem 2 is $\sum_{i=1}^{m}\mathbb{P}(X_{(j)} = i \:|\:X(t) = i) \: \mathbb{P}(X(t)=i)$. The second term in the product can easily be written in terms of $p_{0}, P$ and $t$. But I don't know how to compute the conditional (I tried to think in terms of taboo probabilities with not much luck). Or perhaps there is a better strategy altogether?

Any reference or ideas are welcome. Assume irreducible, aperiodic if that helps.

Post Undeleted by Stefan Kohl, Yemon Choi, Joseph O'Rourke
Post Deleted by CommunityBot
Source Link

Order statistic of Markov chain sample path and related probabilities

Consider a 1D sample path, denoted as $\{X(1), ..., X(t), ..., X(n)\}$, generated from a discrete time finite state (time homogeneous) Markov chain over states $\{1,...,m\}$, with transition probability matrix $P$, and initial occupation probability vector $p_{0}$. Let $X_{(1)} \leq ... X_{(j)} \leq ... \leq X_{(n)}$ denote the non-descending rearrangement of the sample path. As is well known, $X_{(j)}$ is called the $j$-th order statistic for this sample path of length $n$.

Problem 1. For $i,j\in\{1,...,m\}$, I want to compute the probability mass function (pmf) for $X_{(j)}$, that is, $\mathbb{P}\left(X_{(j)} = i\right)$. This probability for the easier i.i.d. case is well known, see for example p.23, left column top, here. How to do this for the Markov case?

Problem 2. Compute $\mathbb{P}(X(t) = X_{(j)})$.

Using total probability and Bayes rule, the probability for Problem 2 is $\sum_{i=1}^{m}\mathbb{P}(X_{(j)} = i \:|\:X(t) = i) \: \mathbb{P}(X(t)=i)$. The second term in the product can easily be written in terms of $p_{0}, P$ and $t$. But I don't know how to compute the conditional (I tried to think in terms of taboo probabilities with not much luck). Or perhaps there is a better strategy altogether?

Any reference or ideas are welcome. Assume irreducible, aperiodic if that helps.