Orientability of moduli space and determinant bundle of ASD operator

Question

Setting

In instanton gauge theory, given a $G$-principal bundle $P\to X^4$, the orientability of the moduli space of ASD connections $$\mathcal{M}_k = \{A \in L^{2}_{k}(X, \Lambda^1 \otimes\mathrm{ad}(P))\ | \ F_A^+ = 0\}/\mathcal{G}_{k+1}$$ (the subscript is a Sobolev parameter) is equivalent to the triviality of the determinant bundle of (the family of) operator(s) $$D_A : L^{2}_{k}(X, \Lambda^1 \otimes\mathrm{ad}(P)) \to L^{2}_{k-1}(X,(\Lambda^+ \oplus \Lambda^0)\otimes\mathrm{ad}(P))\\$$ $$D = d_A^+\oplus d_A^*.$$ This is a line bundle over the space of connections modulo gauge $\mathcal{B}_k$ (suppose we don't have any reducible for simplicity), $$\det D\to \mathcal{B}_k$$ where the fiber is given by $\det D_A := \Lambda^{\dim \ker D_A} \ker D_A \otimes \Lambda^{\dim \mathrm{coker} D_A} \mathrm{coker} D_A$

Suppose that $\mathcal{M}_k$ is smooth. We know that the tangent space at $[A] \in \mathcal{M}_k $, is given by $$T_{[A]}\mathcal{M}_k\simeq \ker d_A^+/ Im(d_A)$$ thus it seems to me that orientability should be equivalent to the bundle $$\bigsqcup_{[A] \in \mathcal{M}_k}\Lambda^{top}\ker d_A^+/ Im(d_A)\to \mathcal{M}_k$$ being trivial. This bundle is a priori different from $\det D$. Indeed, even if $\ker d_A^+/ Im(d_A) = \ker D_A$, in $\det D$ we are tensoring with the top power of the cokernel. What is the point of the cokernel tensor factor?

I know that we want $S_1 = \ker d_A|_{L_k^2(\Lambda^0\otimes \mathrm{ad}P)}$ and $S_2 = \mathrm{coker}\ d_A^+\subset L_k^2(\Lambda^+\otimes \mathrm{ad}P)$ to be trivial in order to have a smooth neighborhood at $[A]$. And that $\mathrm{coker}D_A > S_1 + S_2$ but maybe $\mathrm{coker}D_A$ is not trivial, so we don't have that $\det D_A = \Lambda^{top}\ker d_A^+/ Im(d_A) $ (maybe I am wrong).

Edit after M. Miller's answer In this case $\mathrm{coker}D_A = S_1 \oplus S_2 = \mathrm{coker} d_A^* \oplus \mathrm{coker} d_A^+$ (and consequently $[A] \in \mathcal{M}$ has a smooth neighbourhood iff $\mathrm{coker} D_A = (0)$). Indeed $$S_1 = \ker d_A|_{L_k^2(\Lambda^0\otimes \mathrm{ad}P)} \simeq \mathrm{coker} \ d_A^*.$$ Moreover
$$(\ker d_A^*)^\perp = Im \ d_A|_{L_k^2(\Lambda^0\otimes \mathrm{ad}P)} < \ker d_A^+$$ therefore $d_A^+ (\ker d_A^*) = Im \ d_A^+$ and consequently $$D_A (\ker d_A^*) = Im \ d_A^+ \quad \Rightarrow Im \ D_A = Im \ d_A^+ \oplus Im \ d_A^*$$

Answer 1

You want a line bundle that's well-defined over the entire space of irreducible connections $\mathcal B^*$; this is how you see, for instance, that a choice of orientation at one point in the moduli space induces an orientation at every point (even if the moduli space is not connected). Similarly, one proves orientability using this line bundle on the whole configuration space.

The general recipe, taking a space of Fredholm operators and putting a line bundle over it, is called the determinant line bundle construction and was introduced by Quillen in the context of Riemann surfaces here. As you say, the fiber above the Fredholm operator $A$ is $\Lambda^\text{max} \text{ker}(A) \otimes \Lambda^{\text{max}} \text{coker}(A)$.

If you try to define this only using the $\Lambda^{\text{max}} \ker(A)$ factor you do not get a locally trivial line bundle, unless you restrict to the locus of surjective operators. Because not every irreducible connection has surjective $(D_A, d_A^*)$, you won't get a line bundle over all of $\mathcal B^*$. But along the locus of surjective operators, $\det(A) \cong \Lambda^{\text{max}} \ker(A)$.

In particular, the bundle you say is `a priori different from $\det D$' is not: because you have assumed that $\mathcal M$ is smooth, you have assumed that $\text{coker}(D_A) = 0$ for all $A \in \mathcal M$. (This is precisely what it means for $A$ to be cut out transversely.) So the cokernel factor is not present.

Note that throughout this answer I have restricted to irreducible connections; otherwise some more care should be taken in defining the determinant line bundle (if you're not careful, you would see 'jumps' along the reducible locus.)