This exact game was studied and published in 1970: Henson's "Winning Strategies for the Ideal Game".
In particular, if $R$ is a single-variable polynomial over any PID then the first player wins. Generally, if $R$ is an integral domain with an element $x$ such that $R/(x)$ is a PID but not a field then the first player wins, and the second player wins the corresponding $R/(x^2)$ game.
