Let G be a Lie group acting on a manifold M And say that there exists a point $m \in M$ such that the stabilizer of the point is non trivial. I am unable to understand how the quotient M/G fails to be a manifold. More precisely, how is a singularity created at the point $m \in M$ while constructing the quotient. Can anybody help me understand this?

$\begingroup$ If the stabilizer of a point is nontrivial, the quotient may or may not be a manifold. For example, the former occurs for the $SO(3)$action on $S^2$ with $SO(2)$ stabilizers, and the latter happens for the $Z^2$action on $R^3$, where the quotient is a cone over $RP^2$, which is not a manifold. $\endgroup$ – Igor Belegradek Sep 14 '11 at 22:12

$\begingroup$ Consider the action of $\mathbb Z_2$, a zerodimensional Lie group, on $\mathbb R^2$ such that the non trivial element acts as $(x,y)\mapsto (x,y)$. $\endgroup$ – Mariano SuárezÁlvarez Sep 14 '11 at 22:13

1$\begingroup$ Take $S^1$ with the action of $\mathbb{Z}/2$ by conjugation. The quotient is the interval $[1,1]$. $\endgroup$ – Fernando Muro Sep 14 '11 at 22:14

1$\begingroup$ (By the way, your title is quite at odds with your question!) $\endgroup$ – Mariano SuárezÁlvarez Sep 14 '11 at 22:14

1$\begingroup$ Mariano: the quotient of the diagonal circle action on $\mathbb C^{n+1}$ is a cone over $CP^n$, which isn't a manifold if $n>1$. $\endgroup$ – Igor Belegradek Sep 14 '11 at 23:47
A general nonsense answer to your question would be the orbit decomposition theorem. Given a compact lie group $G$ acting on a manifold $M$, it describes a stratification of $M$ by orbit types.
The main theorem is that if $p \in M$, and $G_p$ is the stabilizer of $p$, then there is a $G$equivariant diffeomorphism between a neighbourhood of $G.p$ (the orbit of $p$) in $M$ and
$$V \times_{G_p} G$$
where $V$ is a vector space and $G_p$ acts linearly on $V$, i.e. there is a representation $G \to Aut(V)$. So $V \times_{G_p} G$ denotes the quotient of $V \times G$ by the diagonal action of $G_p$. $V \times_{G_p} G$ is an equivariant tubular neighbourhood of $G.p$, and the projection map $V \times_{G_p} G \to G.p$ is simply projection onto the 2nd factor $V \times_{G_p} G \to G/G_p$ followed by the identification $G/G_p \equiv G.p$ given by the action.
So the underlying reason for $M/G$ to have singularities is that $(V\times_{G_p} G)/G \simeq V/G_p$ i.e. the action of $G_p$ on $V$ tends to have a singular quotient. Igor gave an example of this in his comment.

$\begingroup$ Ah, thanks Ryan! That clarifies a lot. I worked out the examples above and now have a much better understanding. Thanks again! $\endgroup$ – Varun Sep 16 '11 at 9:29
Complementing Ryan's answer: we can assume that $G$ preserves a Riemannian metric on $M$ (since $G$ is compact, we can take just any metric and average it with respect to $G$). Then $V$ from Ryan's posting is the the orthogonal complement in $T_p M$ of the tangent space of the orbit.
The quotient $M/G$ locally looks like $V/G_p$. This will often be singular, but when $M$ is a complex 1manifold and $G$ is a finite group that acts by holomorphic transformations, the quotient will be a smooth manifold: any finite subgroup $H\subset GL_1(\mathbb{C})=\mathbb{C}^*$ is generated by some root of 1, and so $\mathbb{C}/H\cong \mathbb{C}$.

$\begingroup$ Thanks a lot Algori. The title was a little misleading. I actually needed a theoretical understanding of the fact as to why the singularity occurs. I have a better understanding of it now. $\endgroup$ – Varun Sep 16 '11 at 9:38