5
$\begingroup$

As we know that the transition matrix $P$ of a Markov chain with finite space is a stochastic matrix, and from Perron-Frobenius Theorem, we know that the spectral radius of the matrix $P$ is $1$, and the eigenvalue $1$ is simple. I was wondering if we consider ergodic Markov process with continuous spaces (from which we know that there exists a unique invariant measure), what would be the eigenvalue and eigenfunction of the corresponding Markov operator (sometimes called Perron-Frobenius operator)? Is the eigenvalue guaranteed to have module smaller than $1$? Is the eigenvalue simple as well?

$\endgroup$
3
$\begingroup$

With respect to slightly modified versions of your second and third questions (Is the second eigenvalue guaranteed to have module smaller than 1? Is the first eigenvalue simple as well?), the answers are both Yes, if the transition matrix is Harris recurrent and the associated operator self-adjoint and compact.

The case of MC on continuous state spaces $S$ is different from the case of finite state space. First of all, the transition matrix is replaced by a transition kernel $P(x,A)$=probability of reaching $A\subset S$ from the state $x.$ In this case, if there is an invariant distribution $\pi$ and $P$ is $\pi$ irreducible, then $\pi$ is the unique invariant distribution of $P,$ moreover, if $P$ is aperiodic, then for $\pi$-almost all $x$ we have the convergence of $P^n(x,A)$ (as $n\to \infty$) to $\pi(A)$ in the Total Variation Norm.

One can recover a similar version of the finite state space case only when $P$ is Harris recurrent, as the convergence occurs for all $x.$ Under this assumption one can do the following:

  • Define the Hilbert space $L^2(\pi)=\{f: \int |f|^2 d\pi<\infty \}$ with the natural inner product.
  • If $L^2(\pi)$ is separable and it has an orthonormal basis, one can define a linear operator on $L^2(\pi)$ by $Af(x):=\int p(x,y)f(y)dy,$ where $p(x,\cdot)$ is the transition density for $P.$

This operator $A$ will be analogous to your transition matrix $P$ for the finite state case: $1$ is an eigenvalue with eigenfunction $f(y)\equiv 1$ (and it is the unique largest eigenvalue). If you make extra assumptions on your operator $A$ you can get more information of the other eigenvalues, for example, if $A$ is self-adjoint and compact then the eigenvalues are in $[0,1],$ there are at most countably many eigenvalues, and the unique possible accumulation point is the origin. You can even study the spectral gap by approximating the second eigenvalue. The importance of the spectral gap in this setting is similar to the finite space case, measuring the efficiency of algorithms for approximating integrals (with all the applications that this may have). It is usually a hard problem and there is a lot of interest for practical Monte Carlo Markov chains.

You may be also interested in the following references:

  • I. Kontoyiannis and S.P. Meyn. Geometric ergodicity and the spectral gap of non-reversible Markov chains. Probability Theory and Related Fields. (2012).
  • M. Ahues, A. Largillier and B. Limaye. Spectral Computations for Bounded Operators. (2001).
  • G. Helmberg. Introduction to Spectral Theory in Hilbert Space. Elsevier. (2014).
  • Q. Qin, J.P. Hobert and K. Khare. Estimating the spectral gap of a trace-class Markov operator, arXiv
$\endgroup$
  • $\begingroup$ Thanks a lot for the detailed explanation! I am relatively new to this field. According to your explanation, is it true that in continuous space settings, only being irreducible and aperiodic is not enough to answer the two questions I posted, correct? Stronger assumptions, as you mentioned, the transition kernel being Harris Recurrent, is required for such conclusions, right? Thanks a lot! $\endgroup$ – zkq Apr 28 '18 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.