(In this question, all rings and algebras are commutative with identity.)

I have a situation that boils down to the following data: a ring $R$, an $R$-algebra $A$ with a subalgebra $B$ such that $A$ and $B$ are free of finite rank as $R$-modules, and an $R$-algebra homomorphism $B\to R$. (Update: If it helps, you can assume that the quotient $R$-module $A/B$ is also free.) In this way, $A$ and $R$ are both $B$-algebras, and we can form their tensor product: $$\begin{array}{ccc}A\otimes_B R & \leftarrow & R\\ \uparrow & & \uparrow \\ A & \leftarrow &B \end{array}$$

The question is:

Must the resulting homomorphism $R\to A\otimes_B R$ be injective?

I know that the kernel must consist of nilpotents by the following argument: since $A$ is finite as an $R$-module it is integral as an $R$-module, and hence also as a $B$-module. Since integral extensions have the Lying Over property, the map of schemes $\mathrm{Spec}(A)\to \mathrm{Spec}(B)$ is surjective. Surjectivity is preserved by base extension, so the map of schemes $\mathrm{Spec}(A\otimes_B R) \to \mathrm{Spec}(R)$ is surjective. And a morphism of affine schemes has dense image if and only if the kernel of the corresponding ring homomorphism consists of nilpotents.

So under the weaker assumption that $A$ merely be integral over $R$, I already find that the map $R\to A\otimes_B R$ is injective modulo nilpotents in $R$. Does the extra assumption that $A$ and $B$ be free of finite rank as $R$-modules imply that the kernel is $0$?

Update: Here are a couple of similarly plausible statements with counterexamples, in case it helps find a counterexample to the original question:

(False) Suppose $R$ is a ring, $A$ is an $R$-algebra, and $B\subseteq A$ is a subalgebra of $A$, such that $A$ and $B$ are both free as $R$-modules. Let $B\to R$ be an $R$-algebra homomorphism. Then $R\to A\otimes_B R$ is injective.

A counterexample is given by setting $B=R[x]$ and $A=R[x,x^{-1}]$. If we choose the homomorphism $B\to R$ sending $x\mapsto 0$, then the tensor product $A\otimes_B R$ is the zero ring.

The original question asks whether an example of such $A$ and $B$ exists with $A$ and $B$ free of finite rank as $R$-modules.

(False) Suppose $A$ is a ring and $B$ is a subring of $A$ such that $A$ is finitely generated as a $B$-module. For any $B$-algebra $R$, the homomorphism $R\to A\otimes_B R$ is injective.

Here's a counterexample (due to Hendrik Lenstra): let $A=\mathbb{Z}[x]/(x^2-x)$ and $B=\mathbb{Z}[y]/(y^2-2y)$; the homomorphism $B\to A: y\mapsto 2x$ is injective. However, tensoring with the $B$-algebra $B/(2)$, we obtain a non-injective map $B/(2)\to A/(2): y\mapsto 0$. The original question asks for an example in which the map $B\to R$ is a section of an $R$-algebra structure on $B$ making $B$ and $A$ free of finite rank as $R$-modules.