Let $R$ be an integral domain.  If $M$ is an $R$-module such that every tensor power of $M$ over $R$ is $R$-torsion-free, then is $M$ necessarily flat as an $R$-module?  If not, then does this implication hold for $R$-algebras $M$, or at least for $R$-algebras $M$ between $R[X]$ and $K[X]$, where $K$ is the quotient field of $R$?

A while ago David Speyer showed in his nice answer below that the answer is yes if $M$ is finitely generated, but the particular modules I'm interested in ($R$-algebras between $R[X]$ and $K[X]$) are not finitely generated.