Markov Process, Markov Chain

Question

I am trying to explain the differences between the following concepts to someone and I realized I myself am super confused:

Continuous/discrete Markov Process

Continuous/Discrete Markov chains

Markov property : $P\{X_n=i|X_{n-1}=j,X_{n-2}=k,...\}=P\{X_n=i|X_{n-1}=j\}$ ?

I used to think: Every process that has Markov property is a Markov Process. Every Markov process is a Markov chain and every Markov chain is a Markov process.

But it seems crazy now when I think about it, because if they are all the same, why there are different names for them?

And they are continuous (discrete) if their parameter set $T$ is continuous (discrete) regardless of their state space?

I want to start with homogeneous Markov chain and process too. But since I am already too confused and Wikipedia is making me more confused, I prefer to wait till I get these basic definitions straight first.

Also if anyone has any suggestions on how to explain these terms w/o causing any confusion I would appreciate it a lot. (I would also contact my SP teacher to teach him that, coz he has clearly taught us bad)

Thanks a lot

Answer 1

Before adding the Markov property, I think it helps to distinguish between discrete time and continuous time stochastic processes with discrete or continuous state spaces. (This classification of stochastic processes is nicely summarized here.) Suppose that $(S,\Sigma)$ is a measurable space and $(\Omega, \mathcal{F}, P)$ is a given probability space. Then an $S$-valued stochastic process is a family of $S$-valued random variables $X_t$ parametrized by a time-like variable $t \in T$ where $T \subset \mathbb{R}$. The space $S$ is the state space of the process, and it could be discrete or continuous. If $T=\mathbb{N}$ then $X_t$ is called a discrete time stochastic process, and if $T$ is some interval in $\mathbb{R}$, then $X_t$ is referred to as a continuous time stochastic process.

A stochastic process with the Markov property is called either a Markov process or a Markov chain. (It seems more natural to call them the former, but Wikipedia uses the latter.) To be sure, the Markov property states that for any $A \in \Sigma$ and for each $s,t \in T$ with $s<t$, $$ P(X_t \in A \mid \mathcal{F}_s) = P(X_t \in A \mid X_s) $$ where $\mathcal{F}_t$ is the filtration of the process up to time $s$. This definition makes perfect sense for $T$ being a discrete or continuous set of times and for $S$ being a discrete or continuous state space. For Markov processes the above classification of stochastic processes is nicely summarized in the following table. This basic framework can be used to classify all sorts of Markov processes including: random walks, Brownian motions, Poisson processes, birth-death processes, diffusion processes, jump diffusion processes and branching processes. For an intro to Markov processes, check out:

• Kirkwood, J. R. Markov Processes. Vol. 20. CRC Press, 2015.

Answer 2

These terms (Markov chain / Markov process) seems to be mostly interchangeable, especially if one cares to specify whether the time is discrete or continuous. I would prefer "chain" to mean discrete time, and "process" to mean continuous time, but this is not a strict convention.

Also, how these terms are used in the Math subject classification:

http://www.ams.org/msc/msc2010.html?t=&s=markov&btn=Search&ls=Ct