Quasi-coherent sheaves of O_X-algebras

Let $X$ be a quasi-compact scheme and let $\mathcal{A}$ be a quasi-coherent sheaf of $\mathcal{O}_X$-algebras on $X$. $X$ being quasi-compact, we can write $X = U_1 \cup \dots \cup U_n$ with each $U_i$ affine, say $U_i =\textrm{Spec}R_i$, and such that $\mathcal{A}\mid_{U_i} \simeq \widetilde{A_i}$ for some finitely generated $R_i$ algebras $A_i$. Suppose we know that each $A_i$ is the union of its subalgebras which are module finite over $R_i$. Can we say that $\mathcal{A}$ is globally the union of it's subsheaves which are coherent sheaves of $\mathcal{O}_X$-algebras?

It seems that the obvious thing to do would be to glue together coherent algebras over each of the $R_i$'s, but it's not clear to me how this can be done. This question arose from Milne's proof of 'Zariski's Main Theorem' from the beginning of his etale cohomology book.

Under some slightly stronger hypothesis (Noetherian is certainly enough) we may write $\mathcal A$ as the union of its coherent subsheaves. If $\mathcal E$ is a coherent subsheaf, then the subalgebra of $\mathcal A$ that it generates will also be coherent, because this can be tested locally, where it then follows from your assumptions. Thus in this case, $\mathcal A$ is the union of coherent $\mathcal O_X$-algebras.