Let $X$ be a quasi-compact scheme and let $\mathcal{A}$ be a quasi-coherent sheaf of $\mathcal{O}_X$-algebras on $X$. $X$ being quasi-compact, we can write $X = U_1 \cup \dots \cup U_n$ with each $U_i$ affine, say $U_i =\textrm{Spec}R_i$, and such that $\mathcal{A}\mid_{U_i} \simeq \widetilde{A_i}$ for some finitely generated $R_i$ algebras $A_i$. Suppose we know that each $A_i$ is the union of its subalgebras which are module finite over $R_i$. Can we say that $\mathcal{A}$ is globally the union of it's subsheaves which are coherent sheaves of $\mathcal{O}_X$-algebras?

It seems that the obvious thing to do would be to glue together coherent algebras over each of the $R_i$'s, but it's not clear to me how this can be done. This question arose from Milne's proof of 'Zariski's Main Theorem' from the beginning of his etale cohomology book.