This *exact* game was studied and published in 1970: [Henson's "Winning Strategies for the Ideal Game"](https://www.jstor.org/stable/2317018?seq=2).

In particular, if R is a single-variable polynomial over any PID then the first player wins.