If $M$ is paracompact, or more generally $P$ is trivialised over a numerable open cover, this map is indeed the classifying map for $P\to M$. Note that the universal bundle $EG \to BG$ is given by the geometric realisation of $G\rtimes G \to \ast \rtimes G$, since you are assuming $G$ well-pointed (so that $|N (\ast \rtimes G)| \simeq BG$).
So to show that $M\to BG$ as you give is the classifying map, it is enough to show that $P\times_M N \simeq c^\ast EG$, where $N:= |N(P\rtimes G)|$ and $c\colon N\to BG$. Since $M$ is paracompact, $N\to M$ is shrinkable, and hence a homotopy equivalence. Thus given a section, pulling $P\times_M N$ back along this section gives a bundle isomorphic to $P$, as pullback along homotopic maps (in case the identity, and pulling up then back down) gives isomorphic bundles, which gives the result that $M\to BG$ classifies $P$.
Now assuming that we use the k-space product, or that all the spaces involved are locally compact weak Hausdorff, then geometric realisation of simplicial spaces preserves pullbacks. Thus $c^\ast EG$ is given by the geometric realisation of the nerve of $(P\rtimes G) \times_{(\ast\rtimes G)} (G\rtimes G)$, and $P\times_M N$ by the geometric realisation of the nerve of $\underline{P}\times_\underline{M}(P\rtimes G)$. Thus we only need to show that there is an isomorphism of groupoids
$$
(P\rtimes G) \times_{(\ast\rtimes G)} (G\rtimes G) \simeq (P\times G) \rtimes G \to \underline{P}\times_\underline{M}(P\rtimes G) \simeq (P\times_M P) \rtimes G,
$$
where the action in $(P\times G) \rtimes G$ is diagonal, and the action in $(P\times_M P) \rtimes G$ is on the right factor of $P$ only.
The map on the level of objects is just
$$
P\times G \to P\times_M P,
$$
sending $(p,g) \mapsto (p,pg)$, and by principality of the action this is an isomorphism. The map on the level of arrows is
$$
P\times G \times G \to P\times_M P \times G
$$
sending $(p,g_1,g_2) \mapsto (p,pg_1,g_2)$, again an isomorphism. It is not too difficult to see that this is funtorial, and so we get an isomorphism of geometric realisations as desired.
As far as a reference is concerned, Segal's paper Classifying spaces and spectral sequences, Publications Mathématiques de l'IHÉS, 34 (1968), p. 105-112 (Numdam) is probably the best, though he doesn't give the direct argument from above, using instead cocycles for an open cover (if you think of the space P as a cover over which the bundle P trivialises, then his argument can be applied to recover the above).