I'm interested in a continuous-state, discrete-time Markov process. Let the distribution at time $t$ be $f_t(x)$. The update equation has the form \begin{equation} f_{t+1}(x) = \int f_t(x') g(x', x) dx'. \end{equation} I'd like to find an analytic solution/approximation for the steady-state distribution $f(x)$, which satisfies \begin{equation} f(x) = \int f(x') g(x', x) dx'. \end{equation}

If the form of $g$ is known, is there a good strategy for solving for $f$?

I'm more used to having a continuous-time system, where there would have a Fokker-Planck equation that I can attack with all the standard machinery of differential equations. But I can't think of a good strategy here.

Edit: In light of Benoît's comment below, then I would like to reduce the scope of my question. Can anyone point me to examples that are solvable analytically, along with solutions? I just want to see if I can get any inspiration toward a solution, or whether I need to just rely on numerics.

  • 1
    $\begingroup$ Your question seems too general to expect a satisfying answer. In many cases, proving the mere existence of a steady-state distribution is a Theorem by itself. There are context where you can prove a good (e.g. exponential) convergence of $f_t$ to a steady-state $f$, in which case you can approximate $f$ by $f_t$ with large $t$, but in such a generality I do not see what more to say. $\endgroup$ Feb 24 '15 at 12:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.