Why do symmetries of K3 surfaces lie in the Mathieu group $M_{24}$?

Question

I'm having trouble following some steps of this argument from the appendix of Eguchi, Ooguri and Tachikawa's paper Notes on the K3 surface and the Mathieu group M24:

Now let us recall that the cohomology lattice of K3, $$ \Lambda = H^\ast(\textrm{K3}, \mathbb{Z}) $$ is also an even self-dual lattice of dimension 24, but with signature (4, 20). The close connection between $M_{24}$ and the geometry of the K3 surface stems from this fact. Take a K3 surface $S$, and let $G$ its symmetry preserving the holomorphic 2-form. Let $\Lambda^G$ be the part of $\Lambda$ preserved by $G$, and $\Lambda_G$ its orthogonal complement. $\Lambda_G$ is inside the primitive part of $H^{1,1}$, thus it is negative definite. Using Nikulin’s result, it can be shown that $\Lambda_G$ is a sublattice of $N$. Therefore $G$ is a subgroup of $M_{24}$.

1. Why is the orthogonal complement $\Lambda_G$ inside the primitive part of $H^{1,1}$?

2. Why does this imply that $\Lambda_G$ is negative definite?

3. What result of Nikulin is being referred to? (There is no reference to Nikulin in this paper, and no mention of him other than in this sentence.)

4. How can we conclude that $G$ is a subgroup of $M_{24}$?

The authors explain that $N$ is an even self-dual lattice in a 24-dimensional Euclidean space containing the lattice $A_1^{24}$, and they discuss its relation to $M_{24}$, but they don't seem to say how they're treating it as a lattice inside $H^*(\textrm{K3}, \mathbb{Z})$.

Answer 1

This is clearly not the best reference on the subject — I would recommend Mukai's original paper: Finite groups of automorphisms of K3 surfaces and the Mathieu group, Invent. Math. 94 (1988), no. 1, 183-221. Anyway:

1. I suppose that "primitive" refers to a polarization $h\in H^{1,1}$, and that it is assumed the $G$ preserves that polarization. Therefore $h\in \Lambda_G$, so $\Lambda^G$ is orthogonal to $h$, hence contained in $H^{1,1}_{prim}$.

2. On $H^{1,1}_{prim}$ the intersection form is negative definite — this is (part of) the Hodge index theorem.

3. I think this refers to Integer symmetric bilinear forms and some of their geometric applications. Math USSR-Izv. 14 (1979), no. 1, 103-167.

I am not sure about 4 — this should be much clearer in Mukai's paper.