You should assume that each $A_n$ is a flat module over $R$.

Consider $A$ where each $A_n=R$ with basis element $x_n$, with multiplication $x_n x_m = (2-q)^{nm} x_{n+m}$. Since $nm = ((n+m)^2 - n^2 - m^2)/2$, $A$ is associative. The algebra $A' = A/(1-q)$ is simply $\mathbf{C}[x_1]$, so finitely generated, but $A$ is not finitely generated because $A/(2-q)$ has trivial multiplication.