Suppose $D$ is a non-empty set and $\{ R_i : i \in \mathbb{N} \}$ is a family of binary relations on sequences over $D$ so that $R_i \subseteq D^i \times D^i$. Let $R_\omega \subseteq D^\omega \times D^\omega$ be a relation that holds between two denumerable sequences $x$ and $y$ over $D$ iff for all $n\in \mathbb{N}$, $x_1,\ldots,x_n R_n y_1,\ldots,y_n$.

Now for every ordinal $n \leq \omega$, there is a Kripke frame $F_n = (D^n, R_n)$. I would like to show that any formula of (standard propositional) modal logic is valid in $F_\omega$ iff it is valid in all $F_n$ with finite $n$.

For example, clearly $R_\omega$ is reflexive iff all $R_n$ with finite $n$ are reflexive, so $\Box p \to p$ is valid in $F_\omega$ iff it is valid in all $F_n$ with finite $n$. But is this true for all formulas? If so, how would I go about showing it?