In the paper "Self-duality in Four-dimensional Riemannian Geometry" (1978), Atiyah, Hitchin and Singer present a proof that the space of self-dual irreducible Yang-Mills connections is a Hausdorff manifold, and if it is not the empty set, then the dimension is given by $$p_1(\text{Ad}(P))-\frac{1}{2}\dim G(\chi(M)-\tau(M))$$ Where $\chi(M)$ is the Euler characteristic and $\tau(M)$ the signature.
EDIT: It turns out the original paper contained an error/typo. It should in fact be $$2p_1(\text{Ad}(P))-\frac{1}{2}\dim G(\chi(M)-\tau(M))$$ End of edit.
Although I would love to be able to understand the full paper, I am not in a position to be able to do so yet, I am only trying to understand the computation of this dimension, because I am interested in some applications of the Atiyah-Singer index theorem.
To compute this dimension, the following is utilised in the paper:
Let $D:\Gamma(V_-\otimes E)\to\Gamma(V_+\otimes E)$ be the Dirac operator for a spinor bundle with values in some auxiliary bundle $E$. By the index theorem,
$$\text{ind}(D)=\int_M\text{ch}(E)\widehat{A}(M)$$
In dimension four, we have $\widehat{A}(M)=1-\frac{1}{24}p_1(M)$ (but where is this used?). For the proof, we take $E=V_-\otimes\text{Ad}(P)$. Then $\text{ch}(E)=\text{ch}(\text{Ad}(P))\text{ch}(V_-)$. So far, so good. I lose track in the following computation:
$$\text{ind}(D)=\int_M\text{ch}(\text{Ad}(P))\text{ch}(V_-)\widehat{A}(M)\\
\color{red}{=p_1(\text{Ad}(P))+\dim G(\text{ind}(D'))}=\\
p_1(\text{Ad}(P))-\frac{1}{2}\dim G(\chi-\tau)$$
Where $D':\Gamma(V_+\otimes V_-)\to\Gamma(V_-\otimes V_-)$. I have been trying to find a result that explains the red coloured part of the equation, because this step seems completely non-trivial, and in spite of that, it is not elaborated upon within the paper at all, and I am not able to find any sources that explain this step. In Index of Dirac operator and Chern character of symmetric product twisting bundle the accepted answer seems to give an answer that goes some way towards explaining how this result is obtained, in a very particular case. However, I am not very experienced in this area and I don't know how to generalise the result to an arbitrary principal $G$-bundle. I am looking for an explanation of the above, whether someone is able to provide their own response or a reference. Either one would be greatly appreciated.