The answer to your first question is a resounding no. An example (among many) is given by $X=\mathrm{Spec} k$, $R=k[x]/(x^2)$ and $M=k$ considered as an $R$-module through the $k$-algebra homomorphism given by $x\mapsto 0$. As for the second question, the reason is that in general there are correction terms to this formula coming from the failure of $M$ being locally projective. In fact there is a long exact sequence $$ 0\rightarrow H^0(X,\mathcal{E}\mathrm{xt}^1_R(M,M))\rightarrow \mathrm{Ext}^1_R(M,M)\rightarrow H^1(X,\mathcal{H}\mathrm{om}_R(M,M)) \rightarrow H^0(X,\mathcal{E}\mathrm{xt}^2_R(M,M)), $$ where the $\mathcal{E}\mathrm{xt}^i_R(M,M)$ are the sheaves of Ext-classes (their stacks at $x$ are the $\mathrm{Ext}^i_{R_x}(M_x,M_x)$). Hence, you need something like (possibly something a little bit weaker) the vanishing $\mathrm{Ext}^i_{R_x}(M_x,M_x)$ for $i=1,2$ which in turn are implied by (though not implying) the local $R$-projectivity of $M$.