What you want is statement 4.2 in Kropholler's paper, "[On groups of type $(FP)_\infty$](https://doi.org/10.1016/0022-4049(93)90136-H)" (1993). This states that your colimit is zero if and only if the module $M$ has finite projective dimension over $R$. No finite generation hypotheses are involved. His (and my) notation for this is $\widehat{\operatorname{Ext}}^0_R(M,M)$.