I'm working in reliability field and have seen papers written on the topic like process of failures when systems are functioning under unobservable (or observable) Markov-like environment, i.e. probability to fail is dependent on the state of environment. This state is described as discrete-state discrete-time homogeneous Markov chain. In mathematical notation it would be like this:
$ P\left [ X_{t}=k | Y_{t}=i \right ]=\pi _{k}\left ( i \right ) $;
where $ X _{t} $ is a binary random failure process with possible states 0 (failed) and 1 (working). And $Y _{t}$ is a Markov process at the moment $t$ being in the $i ^{th}$ state.

My question:

Is it possible (or even reasonable) to extend mentioned model? For example, from discrete-state to continuous-state Markov model? Is there any literature (I havent found yet) about continuous stochastic conditional processes. I suppose it is not so trivial, because for continuouse stochastic process statements like $Y_{t}=i$ are meaningless.

  • $\begingroup$ It is easy to extend: for each $t$ there is a conditional law of $X_t$ given $Y_t$. $\endgroup$ – Stéphane Laurent Feb 28 '12 at 15:36
  • $\begingroup$ @ Stéphane Laurent Could recommend some literature (papers) on this the conditional stochastic processes topic. Because all I can find is just short notes in textbooks and no deeper analysis. I would be greatfull for it. $\endgroup$ – Tomas Feb 29 '12 at 9:44
  • $\begingroup$ The formula $P(X_t=k\mid Y_t=i)=\pi_k(i)$ can be reformulated as $P(X_t=k\mid Y_t)=\pi_k(Y_t)$, and this formula makes perfectly good sense, no matter what the distribution of $Y_t$. See en.wikipedia.org/wiki/Conditional_expectation#Formal_definition and en.wikipedia.org/wiki/…. $\endgroup$ – Jason Swanson Mar 15 '12 at 20:33

This question is related to the topic of stochastic filtering theory. See e.g. the following monographs * Bucy, Joseph - Filtering for stochastic processes with applications to guidance * Bain, Crisan - Fundamentals of stochastic filtering * Kallianpur - Stochastic filtering theory

Explicit solutions exist for the linear case (Kalman-Bucy filter). For the nonlinear case the situation is more complicated. See also the wikipedia page http://en.wikipedia.org/wiki/Kalman_filter#Kalman.E2.80.93Bucy_filter


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.