I'm interested in a continuous-state, discrete-time Markov process. Let the distribution at time $t$ be $f_t(x)$. The update equation has the form \begin{equation} f_{t+1}(x) = \int f_t(x') g(x', x) dx'. \end{equation} I'd like to find an analytic solution/approximation for the steady-state distribution $f(x)$, which satisfies \begin{equation} f(x) = \int f(x') g(x', x) dx'. \end{equation}

If the form of $g$ is known, is there a good strategy for solving for $f$?

I'm more used to having a continuous-time system, where there would have a Fokker-Planck equation that I can attack with all the standard machinery of differential equations. But I can't think of a good strategy here.

Edit: In light of Benoît's comment below, then I would like to reduce the scope of my question. Can anyone point me to examples that are solvable analytically, along with solutions? I just want to see if I can get any inspiration toward a solution, or whether I need to just rely on numerics.

  • 1
    $\begingroup$ Your question seems too general to expect a satisfying answer. In many cases, proving the mere existence of a steady-state distribution is a Theorem by itself. There are context where you can prove a good (e.g. exponential) convergence of $f_t$ to a steady-state $f$, in which case you can approximate $f$ by $f_t$ with large $t$, but in such a generality I do not see what more to say. $\endgroup$ Feb 24, 2015 at 12:26


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.