Under the assumption that a *countable direct product of modules over $R$* means a direct product of countably many modules over $R$, I answer OP's question when $R$ is **Noetherian**. In addition, I outline a remark for $R$ an arbitrary **countable** ring with identity.

**Claim 1.** Let $R$ be a commutative ring with identity. If $R$ is Noetherian then the following are equivalent:

$(i)$ Every direct product of countably many projective modules over $R$ is projective.

$(ii)$ $R$ is Artinian.

*Proof*. The implication $(ii) \Rightarrow (i)$ is given by Stephen Chase's Theorem [1, Theorems 3.3 and 3.4]. Assume that $(i)$ holds. Then $M \Doteq R^{\aleph_0}$ is a projective module. If $R$ is moreover connected, then $M$ is free over $R$ by [2, Corollary 4.5]. $R$. By John O'Neill's result [3, Lemma 1.1], the ring $R$ is then Artinian. If $R$ is not connected, it is the direct product of finitely many non-trivial connected rings $R_i$. By hypothesis, the module $M_i \Doteq R_i^{\aleph_0}$ is projective over $R$, and hence over $R_i$, for every $i$. As $R$ is a product of finitely many Artinian rings, it is Artinian.

For an arbitrary countable ring, the following weaker result holds.

**Claim 2.** If $R$ is a countable ring with identity such that $R^{\aleph_0}$ is projective over $R$, then $R$ is left perfect.

*Proof.* Apply [1, Theorem 3.1] and Bass's characterization of left perfect rings.

Coming back to the commutative setting, we get

**Corollary.**
Let $R$ be an integral domain. If $R$ is countable then the following are equivalent:

$(i)$ Every direct product of countably many projective modules over $R$ is projective.

$(ii)$ $R$ is a field.

*Proof.* Any element of a perfect commutative ring is either a unit or a zero divisor. The result is then a direct consequence of Claim 2.

[1] S. Chase, "Direct product of modules", 1960.

[2] H. Bass, "Big projective modules are free", 1962.

[3] J. O'Neill, "When a ring is an $F$-ring", 1993.