Why does differencing create wide-sense stationary time series?

Question

In time series analysis, a common assumption made is that the series is wide-sense stationary, ex. that it has time invariant mean and covariance. However, as this is often not the case in real life, a common approach is to take the difference of the time series:

D_(i) = X_(i) - X_(i-1).

If it doesn't work, then you can do it again.

In fact, the AutoRegressive Integrated Moving Average (ARIMA) Model for time series forecasting has a parameter for the number of times to take the difference.

My question: why does differencing work in reducing non-stationary time series to time series? Does it always work? What is the theory here?

Thanks!

Answer 1

I don't know why that is a common practice, but it makes sense in the case of a Brownian motion with drift $X_t=\sigma W_t+\nu t$. Then we have $$ \mathbb E X_{t+1}=\nu(t+1) $$ which is not time invariant, but $$ \mathbb E (X_{t+1}-X_t)=\nu $$ which is time invariant. If the drift was of higher order, $\nu t^k$, then you could achieve the same effect by taking the difference $k$ times.

As for the variance, it is $\sigma^2 t$ in this case, and after taking the difference once, $$ \text{Var}(X_{t+1}-X_t)=\sigma^2\text{Var}(W_{t+1}-W_t)=\sigma^2 $$ which is time invariant.