I'm working in reliability field and have seen papers written on the topic like process of failures when systems are functioning under unobservable (or observable) Markov-like environment, i.e. probability to fail is dependent on the state of environment. This state is described as discrete-state discrete-time homogeneous Markov chain. In mathematical notation it would be like this:
$ P\left [ X_{t}=k | Y_{t}=i \right ]=\pi _{k}\left ( i \right ) $;
where $ X _{t} $ is a binary random failure process with possible states 0 (failed) and 1 (working). And $Y _{t}$ is a Markov process at the moment $t$ being in the $i ^{th}$ state.

My question:

Is it possible (or even reasonable) to extend mentioned model? For example, from discrete-state to continuous-state Markov model? Is there any literature (I havent found yet) about continuous stochastic conditional processes. I suppose it is not so trivial, because for continuouse stochastic process statements like $Y_{t}=i$ are meaningless.

  • $\begingroup$ It is easy to extend: for each $t$ there is a conditional law of $X_t$ given $Y_t$. $\endgroup$ Feb 28, 2012 at 15:36
  • $\begingroup$ @ Stéphane Laurent Could recommend some literature (papers) on this the conditional stochastic processes topic. Because all I can find is just short notes in textbooks and no deeper analysis. I would be greatfull for it. $\endgroup$
    – Tomas
    Feb 29, 2012 at 9:44
  • $\begingroup$ The formula $P(X_t=k\mid Y_t=i)=\pi_k(i)$ can be reformulated as $P(X_t=k\mid Y_t)=\pi_k(Y_t)$, and this formula makes perfectly good sense, no matter what the distribution of $Y_t$. See en.wikipedia.org/wiki/Conditional_expectation#Formal_definition and en.wikipedia.org/wiki/…. $\endgroup$ Mar 15, 2012 at 20:33

2 Answers 2


This question is related to the topic of stochastic filtering theory. See e.g. the following monographs * Bucy, Joseph - Filtering for stochastic processes with applications to guidance * Bain, Crisan - Fundamentals of stochastic filtering * Kallianpur - Stochastic filtering theory

Explicit solutions exist for the linear case (Kalman-Bucy filter). For the nonlinear case the situation is more complicated. See also the wikipedia page http://en.wikipedia.org/wiki/Kalman_filter#Kalman.E2.80.93Bucy_filter


What you're looking for is the Wonham Filter. In this setting, $(X_t)_{t\geq 0}$ is a time homogeneous Markov chain defined on a probability space $(\Omega,\mathcal{F},\mathbb{P})$ taking values in a finite state space $S\triangleq \left\{s_n\right\}_{n=1}^N$ governed by the coupled system: $$ \begin{aligned} p_t^n & \triangleq \mathbb{P}(X_t \in s_n)\\ \frac{dp_t}{dt} & = Q p_t,\\ Y_t & = \int_0^t X_s\beta^{\top}ds + W_t \end{aligned} $$ for some Brownian motion defined on the same probability space as $(X_t)_{t\geq 0}$ and where $\beta\in \mathbb{R}^N$ and $Q$ is an $N\times N$ matrix (typically called the "Q-Matrix") and satisfies: $$ \sum_{i=1}^N q_{i,j}=0, \mbox{ and }q_{i,j}\geq 0 \mbox{ for }i\neq j. $$

You can find the solution to this filtering problem in Theorem 22.3.5 of Sam Cohen's book, as well as many other useful facts like occupation times..in Chapter 22.3.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .