What you want is statement 4.2 in Kropholler's paper, "On groups of type $(FP)_\infty$" (1993). This states that your colimit is zero if and only if the module $M$ has finite projective dimension over $R$. No finite generation hypotheses are involved. His (and my) notation for this is $\widehat{\operatorname{Ext}}^0_R(M,M)$$\smash{\widehat{\operatorname{Ext}}}^0_R(M,M)$.