$R$ can be arbitrary. Assume that $H^\ast(Y)$, as a module for $H^\ast(X)$, is free with a finite basis (or even just a basis whose degrees tend to infinity). Suppose also that $X$ is simply connected. I believe that then the answer is "yes".
We may as well assume that $Y\to X$ is a fibration, so that $W$ can be taken to be the strict fiber product $Y\times_X Z$.
The desired statement is that the canonical map $$ H^\ast(Y)\otimes_{H^\ast(X)} H^\ast(Z)\to H^\ast(Y\times_X Z) $$ is an isomorphism, for every $Z$ and every map $Z\to X$. I will consider the more general statement that the canonical map $$ H^\ast(Y)\otimes_{H^\ast(X)} H^\ast(Z,A)\to H^\ast(Y\times_X Z, Y\times_X A) $$ is an isomorphism, for every pair of spaces $(Z,A)$ and every map $Z\to X$.
We may assume that $(Z,A)$ is a CW pair, and by an inverse limit argument we may assume that it is a finite CW pair, and by a five-lemma argument we may reduce to the case when the pair is $(D^n,S^{n-1})$ for some $n\ge 0$. Another five-lemma argument and induction over $n$ reduces us to the case when $Z$ is a point and $A$ is empty. To put it another way, as functors of $(Z,A)$ both sides are cohomology theories, so the map is an isomorphism for all $Z$ if it is so for a point. (We are using that $H^\ast(Y)$ is flat over $H^\ast(X)$.)
So the task is to show that, under our assumptions, the cohomology of the fiber of $Y\to X$ is the tensor product $H^\otimes(Y) _{H^\ast(X)} R$. I have not thought this through, but it seems that it should follow without much trouble from the Serre spectral sequence of $Y\to X$.