Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

I'm working in reliability field and have seen papers written on the topic like process of failures when systems are functioning under unobservable (or observable) Markov-like environment, i.e. probability to fail is dependent on the state of environment. This state is described as discrete-state discrete-time homogeneous Markov chain. In mathematical notation it would be like this:
$ P\left [ X_{t}=k | Y_{t}=i \right ]=\pi _{k}\left ( i \right ) $;
where $ X _{t} $ is a binary random failure process with possible states 0 (failed) and 1 (working). And $Y _{t}$ is a Markov process at the moment $t$ being in the $i ^{th}$ state.

My question:

Is it possible (or even reasonable) to extend mentioned model? For example, from discrete-state to continuous-state Markov model? Is there any literature (I havent found yet) about continuous stochastic conditional processes. I suppose it is not so trivial, because for continuouse stochastic process statements like $Y_{t}=i$ are meaningless.

share|improve this question
It is easy to extend: for each $t$ there is a conditional law of $X_t$ given $Y_t$. –  Stéphane Laurent Feb 28 '12 at 15:36
@ Stéphane Laurent Could recommend some literature (papers) on this the conditional stochastic processes topic. Because all I can find is just short notes in textbooks and no deeper analysis. I would be greatfull for it. –  Tomas Feb 29 '12 at 9:44
The formula $P(X_t=k\mid Y_t=i)=\pi_k(i)$ can be reformulated as $P(X_t=k\mid Y_t)=\pi_k(Y_t)$, and this formula makes perfectly good sense, no matter what the distribution of $Y_t$. See en.wikipedia.org/wiki/Conditional_expectation#Formal_definition and en.wikipedia.org/wiki/…. –  Jason Swanson Mar 15 '12 at 20:33
add comment

1 Answer

up vote 2 down vote accepted

This question is related to the topic of stochastic filtering theory. See e.g. the following monographs * Bucy, Joseph - Filtering for stochastic processes with applications to guidance * Bain, Crisan - Fundamentals of stochastic filtering * Kallianpur - Stochastic filtering theory

Explicit solutions exist for the linear case (Kalman-Bucy filter). For the nonlinear case the situation is more complicated. See also the wikipedia page http://en.wikipedia.org/wiki/Kalman_filter#Kalman.E2.80.93Bucy_filter

share|improve this answer
add comment

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.