Let $\xi_0 \in L^2(A)$ denote the cyclic vector corresponding to the identity in $A$, and let $P_0$ denote the rank-one projection corresponding to $\xi_0$. Then we clearly have $xP_0 = P_0 x = 0$ for all $x \in M$, and hence $M \subset P_0^\perp B(L^2(A)) P_0^\perp$. I claim that we actually have equality. This should be a well known fact to experts in II$_1$ factors (one just needs that $A$ is a unital $*$-algebra which generates a II$_1$ factor), however I don't know a reference off hand so I'll give a proof instead.

To see that $P_0^\perp \in M$ note that if $u \in A$ is a unitary then the spectral projection of $1 - L_{u}R_{u^*}$ corresponding to $\mathbb C \setminus \{ 0 \}$ is contained in $M$. The supremum of these projections over all $u$ is equal to $P_0^\perp$ since $\tilde A := L(A)''$ is a factor.

Note that the representations $L$ and $R$ extend to normal commuting representations of $\tilde A$ (for which I will use the same notation), and it is then easy to show that in the definition of $B$ we may allow $a_i$ and $b_i$ to be in $\tilde A$.

Note also that if $A_0 \subset \tilde A$ is a von Neumann subalgebra, and if $Q$ denotes the projection onto the closure of $(A_0' \cap \tilde A) \xi_0$, then $Q \in \mathbb CP_0 \oplus M$. This follows from the observation that for $\eta \in L^2(A)$ we have that $Q\eta$ is the unique element of minimal norm in the convex closure of $\{ L_uR_{u^*} \eta \mid u \in \mathcal U(A_0) \}$. Hence, $Q$ is in the weak operator topology convex closure of the set $\{ L_uR_{u^*} \mid u \in \mathcal U(A_0) \} \subset \mathbb CP_0 \oplus M$.

In particular, if $p \in \mathcal P(\tilde A)$ is a projection and we set $A_0 = ( \mathbb Cp \oplus \mathbb Cp^\perp )' \cap \tilde A$, then since $\tilde A$ is a factor it follows that the rank one projection onto $(p - \tau(p)) \xi$ is contained in $M$.

Suppose now that we have a self-adjoint operator $T \in M' \cap P_0^\perp B(L^2(A)) P_0^\perp$. Then for $p \in \mathcal P(\tilde A)$ a non-zero projection, we have shown that there exists $\lambda_p \in \mathbb R$ so that $T( p - \tau(p) )\xi_0 = \lambda_p (p - \tau(p))\xi_0$. If $q \in \mathcal P(\tilde A)$ then we have
\begin{align}
\lambda_p \langle (p - \tau(p))\xi_0, (q - \tau(q))\xi_0 \rangle
&= \langle T(p - \tau(p))\xi_0, (q - \tau(q))\xi_0 \rangle \\
&= \langle (p - \tau(p))\xi_0, T(q - \tau(q))\xi_0 \rangle \\
&= \lambda_q \langle (p - \tau(p))\xi_0, (q - \tau(q))\xi_0 \rangle.
\end{align}
Since $\tilde A$ is a factor, any pair of non-zero projections has a third projection so that these inner-products are non-zero. Hence, $\lambda := \lambda_p = \lambda_q$ for all non-zero projections $p, q \in \mathcal P(\tilde A)$. The span of projections is norm dense in $\tilde A$ by the spectral theorem, hence $T(x - \tau(x) ) \xi_0 = \lambda (x - \tau(x))\xi_0$ for all $x \in \tilde A$, and so $T = \lambda P_0^\perp$. Since $T$ was arbitrary, the double commutant theorem then gives the result.