This *exact* game was studied and published in 1970: [Henson's "Winning Strategies for the Ideal Game"](https://www.jstor.org/stable/2317018?seq=2).

In particular, if $R$ is a single-variable polynomial over any PID then the first player wins. Generally, if $R$ is an integral domain with an element $x$ such that $R/(x)$ is a PID but not a field then the first player wins, and the second player wins the corresponding $R/(x^2)$ game.