Let $\{X_k\}$ be an ergodic process. I know that if $f$ is a smooth real valued function then by Birkoff's ergodic theorem, $$\lim_{n\to \infty}\frac{1}{n}\sum_{k=1}^n f(X_k)=\mathbb{E}(f(X_1))\ a.s.$$ Is there any similar result for the $\limsup_n$ or $\liminf_n$ of the sequence $\{f(X_k)\}$, i.e. results which involve expectation? More specifically is there a way to find $$\limsup_n f(X_n),\ \liminf_n f(X_n)$$ almost surely? Though I have asked the question for the general ergodic processes, even results for stationary processes like Gaussian processes will be helpful to me. Also, if they are not available, it will be very kind if someone can give me some references that I can use to find methods to find these results. Thanks in advance.
For ergodic $X_n$, almost surely
$$\limsup_n f(X_n) = \sup_n f(X_n) = \sup \{a \in \mathbb{R} : Pr(f(X_0) > a) > 0\}.$$
In other words, by recurrence, what can happen (with positive probability) will eventually happeninfinitely often.
Update (Sep 4, 2015): You seem most interested in the case of a Gaussian process. Therefore, I should point out what appears to be two incorrect assumptions in your question. You seem to assume that every stationary process is ergodic. This is false, but the converse is true; every ergodic process is stationary. Also you seem to assume that every Gaussian process is stationary. Again this is false.
Consider a Gaussian random variable $X$. Then $X, X, ...$ is stationary and (degenerately) Gaussian, but not ergodic. Moreover, my above formula does not hold; instead $\limsup_n X_n^2 = X_0^2 = X^2$. (I imagine one could construct nondegenerate examples of stationary Gaussian processes where my formula at the top doesn't hold, but I am not certain.)
Further, if $Y_n$ is an i.i.d. sequence of Gaussian random variables with mean 0 and $a_n$ is a sequence of positive real numbers which converges to 0 sufficiently fast, then the sequence $a_n Y_n$ is Gaussian, but it is not stationary. Also, by BorelCantelli, almostsurely $\lim_n a_n Y_n^2 = 0$.
An even better example for my last point would be the following. If $B_t$ is Brownian motion, then the processes $Z_n = B_{1/n}$ is Gaussian but not stationary. Yet, almostsurely, $\lim_n Z_n^2 = 0$.

$\begingroup$ So does this mean that if $\{X_k\}$ is a Gaussian process, $\limsup_n X^2_k=\infty$ and $\liminf_n X^2_k=0$? $\endgroup$ – Samrat Mukhopadhyay Sep 3 '15 at 11:40


$\begingroup$ My last "yes" comment assumed you were referring to ergodic Gaussian processes. I updated end my answer accordingly. $\endgroup$ – Jason Rute Sep 4 '15 at 17:06

$\begingroup$ Thanks for the updates, it cleared some big misconceptions I was having about the ergodic processes. By the way can you kindly direct me to some reference from where I can learn about ergodic processes and stationary processes in depth? $\endgroup$ – Samrat Mukhopadhyay Sep 5 '15 at 14:17