I asked this question on stats.stackexchange.com a little while back but didn't get an answer. It was suggested that I post it here at the time. There appears to be some migratory problems going on over there. Hopefully, this question is seen as appropriate.

Let $X(t)$ be a stochastic process. We say that $X(t)$ is *Nth-order stationary* if for every set of ''times'' $t_1, t_2, \dots, t_N$ we have that the joint cumulative density functions
$$F_{X(t_1),X(t_2),\dots,X(t_N)} = F_{X(t_1 + \tau),X(t_2 + \tau),\dots,X(t_N + \tau)}$$
for all ''time shifts'' $\tau$. This is quite a strong condition, it says that the joint statistics don't change at all as time shifts.

For example, a 1st order stationary process is such that $F_{X(t_1)} = F_{X(t_2)}$ for all $t_1$ and $t_2$. That is, the $X(t)$ are all identically distributed. It is quite easy to see that a 1st order stationary process need not be 2nd order stationary. Simply assign a correlation structure to say $X(t)$, $X(t+1)$, $X(t+2)$ that *does not* correspond to a (symmetric) Toeplitz matrix. That is, in vector form, the covariance matrix of $[ X(t), X(t+1), X(t+3)]$ could be given as
$$\left[\begin{array}{cc}
\sigma^2 & a & b \newline
a & \sigma^2 & c \newline
b & c& \sigma^2
\end{array}\right]$$
for $a,b,c$ distinct. This is now not 2nd order stationary because $E[X(t)X(t+1)] = a$ and, time shifting by 1 we have $E[X(t+1)X(t+2)] = c \neq a$.

In a similar way (presumably) a process that is 1st and 2nd order stationary need not be 3rd order stationary and this leads to my question:

Does somebody have a nice example of a stochastic process that is both 1st and 2nd order stationary, but not 3rd order stationary?

**Motivation:** Some of the material I have been expected to teach recently has included stochastic processes and I feel this is a gap in my knowledge. Although no student has actually asked this question yet, I think it is natural enough for it to be asked at some point and I would like to have a neat and simple example.