Let $A$ be a unitary commutative ring, and let $B$ be an $A$-algebra. We consider the category whose objects are pairs $\textbf{M}=(M,f)$ where $M$ is an $A$-module and where $f$ is a $B$-linear endomorphism of $M\otimes_A B$, and whose morphisms are $A$-linear maps whose scalar extension to $B$ make the obvious diagram commute, i.e. a morphism between $\textbf{M}$ and $(N,g)$ is a $A$-linear morphism $h:M\to N$ such that $g\circ (h\otimes_A 1)=(h\otimes_A 1)\circ f$. Let $\textbf{1}$ be the object $(A,\operatorname{id}_B)$. I want to compute the extension group of $\textbf{1}$ by $(M,f)$ in the latter category. Hereafter, I sketched my computation:
The object $\textbf{P}=(A[X],X)$ is a projective object in this category, hence $0\to \textbf{P}\to \textbf{P}\to \textbf{1}\to 0$ where the first (nontrivial) arrow is multiplication by $1-X$ and the second one is the evaluation at $1$ is a projective resolution of $\textbf{1}$.
Let us compute $\operatorname{Hom}(\textbf{P},\textbf{M})$. We consider the $A$-linear morphism $\operatorname{Hom}(\textbf{P},\textbf{M})\to M$ which maps a morphism $h:A[X]\to M$ to its value $m=h(1)$. By commutativity of the "obvious" diagram $h(X^n)=f^n(m)$ which indicates that $f^n(m)$ belongs to $M$ for all positive $n$. Thanks to this remark, we easily obtain: $$\operatorname{Hom}(\textbf{P},\textbf{M})=\{m\in M~|~\forall n\geq 0:~f^n(m\otimes_A 1)\in M\otimes_A A\}=:f^{\bullet}(M).$$ Hence, $\operatorname{RHom}(\textbf{1},\textbf{M})$ is given by the complex $[f^{\bullet}(M)\to f^{\bullet}(M)]$ in degree $0$ and $1$ where the arrow is $1-f$.
We conclude that $\operatorname{Ext^1(\textbf{1},\textbf{M})}$, which is given by the first cohomology group of this complex, is isomorphic to $\operatorname{coker}(1-f|f^{\bullet}(M))$.
However, let $m\in \operatorname{Hom}_B(B,M\otimes_A B)=M\otimes_A B$. The object $(M\oplus A, \begin{pmatrix} f & m \\ 0 & 1 \end{pmatrix})$ is an extension of $\textbf{M}$ by $\textbf{1}$. Besides, it is equivalent to $(M\oplus A, \begin{pmatrix} f & m' \\ 0 & 1 \end{pmatrix})$ whenever $$\begin{pmatrix} f & m' \\ 0 & 1 \end{pmatrix}\begin{pmatrix} 1 & u \\ 0 & 1 \end{pmatrix}=\begin{pmatrix} 1 & u \\ 0 & 1 \end{pmatrix}\begin{pmatrix} f & m \\ 0 & 1 \end{pmatrix} $$ for some $u\in \operatorname{Hom}_A(A,M)=M$, which boils down to $m'=m+u-f(u)$. Thanks to this argument, it seems that the extension group is rather given by $$\operatorname{coker}(M\stackrel{1-f}{\longrightarrow} M\otimes_A B). $$ Can anyone point me out my mistake? Many thanks!