Reference request: shift invariant measures are (locally exactly) approximable by periodic ones Let $A$ be a finite alphabet, let $S = A^{\mathbb{Z}}$ be the set of bi-infinite sequences of characters from $A$, where $A$ is given the discrete topology and $S$ is given the corresponding product topology, and let $\sigma$ denote the right shift operator.
In order to solve some other problem, I have recently shown the following:

Suppose $\mu$ is a shift invariant probability measure on $S$ (that is, $\mu \circ \sigma = \mu$). Then for every $n \geq 1$, there exists a positive integer $L_n$ and a shift invariant probability measure $\mu_{n}$ such that $\mu_n$ is supported on sequences of period $L_n$ and $\mu_n$'s induced distribution on words of length $n$ is the same as $\mu$'s induced distribution on words of length $n$.

What I'm wondering is whether this is a standard result, or at least already known, and if so, where I can find a reference.
Many thanks!
Note:
Each $\mu_n$ has exactly the same distribution on words of length $n$ as $\mu$. There are counterexamples to this claim when $A$ is infinite, for example, when $A = S^1$.
 A: I don't know wheter this counts as standard, but...
There is a paper by Krystyna Ziemian: 
Rotation sets for subshifts of finite type. Fund. Math. 146 (1995), no. 2, pp. 189--201 
containing some  general results which after some work yield the conclusion. The reduction of your question to Ziemian's framework is fairly standard, but notationally heavy.
Assume that the alphabet has $\alpha$ letters. 
Let $\mu$ be any shift invariant measure. Fix a positive integer $n$. 
Consider a directed graph $G$ with edges labelled by all words of length $n$ and directed edges $u\to v$ between each vertices $u=u_1\ldots u_n$ and $v=v_1\ldots v_n$ such that $u_2\ldots u_n=v_1\ldots v_{n-1}$. Denote the set of edges of $G$ by $E$ and the set of vertices of $G$ by $V$. Note that the edges of $G$ are in the one-to-one correspondence with the words of length $n+1$. There are $N:=\alpha^{n+1}$ edges. For $e\in E$ let $\chi_e\colon E\to\{0,1\}$ be the characteristic function of the edge $e\in E$. Let $\varphi\colon E\to\{0,1\}^N$ be the concatenation of all $\chi_e$'s, that is, for a fixed edge $e_0\in E$ we have $\varphi(e_0)=(\chi_e(e_0))_{e\in E}\in\{0,1\}^N$. Now given an infinite sequence of letters $x$ it  makes sense to define $\varphi(x)=\varphi(x_1\ldots x_{n+1})$ since the first $n+1$ symbols of $x$ determine uniquely an edge of $G$. Then
$$
\lim_{k\to\infty}\frac{1}{k}\sum_{i=1}^{k-1} \varphi(\sigma^i(x))
$$
is the vector whose coordinate given by $e\in E$ describes the limiting frequency of occurrences of the word of length $n+1$ corresponding to $e$ in $x$ (provided the limit exists). In the full shift for every shift invariant measure $\mu$ there is a sequence $x$ such that the above limit exists and equals $(\mu(e))_{e\in E}$, where 
$\mu(e)$ is the measure of the cylinder set determined by $e$ for each $e\in E$. 
This result can be found in Sigmund's paper cited below. In Ziemian's terminology this means that the vector $(\mu(e))_{e\in E}\in [0,1]^N$ belongs to the pointwise rotation set of $\varphi$. Here we follow Ziemian and call a loop in $G$ elementary if it is not a concatenation of two
shorter loops. Clearly, each loop that is not elementary can be written as a
concatenation of two loops, at least one of which is elementary. For each elementary loop $\tau$ of length $L$ in $G$ we can define a periodic sequence $y$ and for that $y$ define its rotation vector $\rho_\tau$ given by
$$
\lim_{k\to\infty}\frac{1}{k}\sum_{i=1}^{k-1} \varphi(\sigma^i(y)).
$$
This simply gives us a vector which is non-zero at $L$ coordinates each equal $1/L$ and corresponds to the periodic shift-invariant measure determined by $y$. Now by Theorem 3.4 of the Ziemian's article cited above the vector $(\mu(e))_{e\in E}\in [0,1]^N$ belongs to the convex hull of $\rho_1,\ldots,\rho_C$ where $C$ is the number of the elementary loops in the graph $G$, and let $\rho_1,\ldots,\rho_C$ are their rotation vectors. This is the desired combination of periodic measures which has the same distribution on finite words of length $n+1$ as $\mu$ as required.
If we only ask about density in the weak star topology of the periodic measures, which means that for each $\epsilon>0$, each shift-invariant measure $\mu$ and any integer $n$ there exists a single periodic orbit with the distribution of finite words of length $n$ which is $\epsilon$-close to the distribution given by $\mu$, then this is a fairly standard result, the oldest reference I know is the paper of Ville:


*

*J. Ville Étude critique de la notion de collectif, vol. 218, Theses françaises de l’entre-deux-guerres, Paris, 1939.
https://eudml.org/doc/192893
Parthasaraty extended it to not necessarily finite alphabets:


*

*K. R. Parthasarathy On the category of ergodic measures. Illinois J. Math. 5 (1961), 648--656. https://projecteuclid.org/euclid.ijm/1255631586
Further generalizations are due to Sigmund:


*

*K. Sigmund Generic properties of invariant measures for Axiom ${\rm A}$ diffeomorphisms. Invent. Math. 11 1970 99--109.


Some more references can be found here: http://arxiv.org/abs/1404.0456
