The confusion is caused by using $R$ to denote two different things.
In Section 13.6, Besse introduces $R$ as a $6$-by-$6$ matrix of (scalar) curvature coefficients, which is the matrix of the linear transformation $\Lambda^2\to\Lambda^2$ induced by the curvature operator with respect to a basis adapted to the standard self-dual-and-anti-self-dual splitting of $\Lambda^2$. When you square this $R$ (which is the same as $R\wedge R$ since the entries are scalars), you will get a symmetric $6$-by-$6$ matrix whose trace is quadratic in the coefficients of the curvature matrix (and, in fact, it will be a positive definite quadratic form on the space of curvature tensors, not what you want at all). Note that you could see that the entries of $A$ and $D$ couldn't be $2$-forms because, if they were, it wouldn't make any sense to subtract $s/12$, which is clearly a scalar. (Actually, it doesn't make any sense anyway because $A$ and $D$ are $3$-by-$3$ matrices, but, never mind. This was only meant to be a heuristic indication of the real formulae anyway.)
Meanwhile, in that passage of Section 13.8, Besse is instead using $R$ to denote a skew-symmetric $4$-by-$4$ matrix of $2$-forms, or, more invariantly, a $2$-form with values in the skew-symmetric endomorphisms of the tangent bundle of $M$. (The entries of $A$, $B$, $C$, and $D$ appears as coefficients of the $2$-form entries in this $R$.) At this point, $R\wedge R$ is a symmetric $4$-by-$4$ matrix of $4$-forms, and the trace is a $4$-form on $M$. As for the appearance of the factors of $2$, that is wrapped up with Besse's choice of the norms on $W_\pm$ as spaces of linear transformations $\Lambda^2_\pm\to\Lambda^2_\pm$. These norms are unique up to a choice of a constant, but one has to make that choice consistently in order for the formulae to work out correctly.
Added remark (at the request of the OP): The point is that, with respect to any local orthonormal coframing, the first Pontrjagin form has to be of the form $p_1 = Q(R)\omega$
where $Q(R)$ is quadratic in the entries of the $6$-by-$6$ matrix $R$ and $\omega$ is the local volume form associated to the coframing. Because of the way that $\mathrm{SO}(4)$ acts on $R$ under change of oriented coframe, $p_1$ must be of the form
$$
p_1 = (c_1\ s^2 + c_2\ |B|^2 + c_3\ |W_+|^2 + c_4\ |W_-|^2)\ \omega
$$
for some universal constants $c_i$ (since $R$ breaks up into 4 inequivalent representation of $\mathrm{SO}(4)$). Now, $p_1$ doesn't depend on a choice of orientation, nor does $s$ nor $B$, but moving to a different orientation will switch $W_+$ and $W_-$ and will also replace $\omega$ by $-\omega$. It follows that the only way the right hand side will be uninfluenced by the choice of orientation is if it is of the form $c\bigl(|W_+|^2-|W_-|^2\bigr)\ \omega$ for some universal constant $c$. The constant $c$ must be positive because $p_1(\mathbb{CP}^2)=3$, and $\mathbb{CP}^2$ has $W_- = 0$. (You can now determine that $4\pi^2c = 1$ by simply doing the integral on $\mathbb{CP}^2$, if you want. Of course, this depends on which norm you chose for $W_\pm$; I'm assuming you choose it to agree with Besse's choice. :) )
I think that one is expected to write out the translation between the two meanings and figure out which choice of norm on $W_\pm$ is meant when one is learning the subject.
Remark: The notation in Besse's Sections 13.6 and 13.8 appears to be drawn, with little modification, from the famous 1978 paper "Self-Duality in Four-dimensional Riemannian geometry" by Atiyah, Hitchin, and Singer. However, the use of $\mathrm{tr}(R\wedge R)$ in the formula in Besse 13.8 seems to have been spliced in from the standard formula for the first Pontrjagin form; it is not in AHS where, instead, a different expression appears. I suspect that Besse also copied the choice of norm for the Weyl tensor from AHS, since they get the very same coefficient.
