Does there exist an integral domain $R$ and an $R$-module $M$ that is not flat over $R$ such that every tensor power of $M$ over $R$ is nonzero and $R$-torsion-free? (If such a domain $R$ exists then it cannot be a Prufer domain.)

Let $R$ be an integral domain. If $M$ is an $R$-module such that every tensor power of $M$ over $R$ is $R$-torsion-free, then is $M$ necessarily flat as an $R$-module? If not, then does this implication hold for $R$-algebras $M$, or at least for $R$-algebras $M$ between $R[X]$ and $K[X]$, where $K$ is the quotient field of $R$?

A while ago David Speyer showed in his nice answer below that the answer is yes if $M$ is finitely generated, but the particular modules I'm interested in ($R$-algebras between $R[X]$ and $K[X]$) are not finitely generated.