Gauge theory construction of moduli of vector bundles Given a smooth projective complex variety $X$, instead of using Mumford's GIT to construct the moduli of rank $n$ topologically trivial vector bundles, we can also take the gauge theory approach. 
To classify all topologically trivial vector bundles is same as to classify all possible complex structures on the topologically trivial vector bundle $X\times \mathbb{C}^n$ module the ambiguity from the choice of basis. And a complex structure is determined by $\bar\partial=\bar\partial_0+\eta$, where $\bar\partial_0$ corresponds to the holomorphic structure as a trivial vector bundle, and $\eta$ is an anti-holomorphic $End(\mathbb{C}^n)$ valued one form, such that $d\eta+\eta\wedge\eta=0$. Let $M$ denote the set of all such $\eta$, and $G$ denote the gauge group with fiber $GL(\mathbb{C}^n)$, then $G$ acts on $M$. Now, set-theoretically the orbits will correspond to the set of isomorphism classes of rank $n$ topologically trivial holomorphic vector bundles, but the quotient topology seems rather bad (non-Hausdorff).
My question is using this approach
(1)Whether stability condition is reflected in this construction?
(2)Can we obtain the same moduli space as using Mumford's GIT?
(3)I guess this is a natural approach. So is there any article or book about this?
The assumption of topologically trivial should not be necessary, just to simplify some notation. We only need to fix the underlying topological vector bundle.
 A: As Charlie Frohman says, for curves this is the Narasimhan-Seshadri correspondonce. For Kahler manifolds of higher dimensions it is called the Hitchin-Kobayashi correspondence, proved by Donaldson-Uhlenbeck-Yau. You search for a Hermitian metric on your vector bundle which is Hermitian-Yang-Mills, i.e. it's curvature form is orthogonal to the Kahler form. Such a metric exists if and only if the bundle is a sum of slope stable bundles. 
In general this can be seen as an infinite dimensional version of GIT vs symplectic reduction. The condition that a metric be HYM says that a certain moment map vanishes for this connection.
To start, I would recommend the following articles:


*

*"The Yang-Mills equations over Riemann surfaces" by Atiyah-Bott.

*"A new proof of a theorem of Narasimhan-Seshadri" by Donaldson.

*"Anti-self-dual Yang-Mills connections over complex algebraic surfaces" by Donaldson. 
A: In the case of a nonsingular algebraic curve, I guess this is the point of Atiyah and Bott's Yang-Mills on Riemann surfaces, or the work of Narasimhan and Seshadri.  Choose a Hermitian pairing on your complex bundle and then search for a unitary connection with central curvature.  There is a really great book by Kobayashi "The Differential Geometry of Complex Vector Bundles" that explores this approach in more generality.
Once you have reduced to unitary connections with central curvature you can understand them as representations of a central extension of the fundamental group of the manifold, which then puts you in a finite dimensional situation with a compact group acting. The quotient corresponds to the geometric invariant theory quotient.
A: Hi Botong, I guess I could say this in person but this is faster.
Given a reductive group acting a finite dimensional Euclidean space, Kempf and Ness proved that the orbits of stable points are precisely the ones on which the norm attains a minimum. This gives a hint of the role of stability in the  analytic construction, 
where now one would like to minimize a suitable energy on the orbits of an infinite dimensional gauge group. This is spelled out in more detail in 
[DK] "The geometry of four manifolds" by Donaldson and Kronheimer, and 
[C] "Flat G-bundles with canonical metrics" by Corlette in addition to the references given above.
Regarding your second question, I suspect that the most meticulous comparison of the
algebraic and analytic constructions can be found in Simpson's papers "Moduli
of representations of the fundamental group I & II".
Addendum: Perhaps I should expand the my answer a bit, since it wasn't terribly enlightening. In very rough terms on the algebraic side one proceeds as follows. 
One first needs to prove that the set of stable vector bundles with fixed topological type $V$ form a bounded family. This already uses the stability condition in an essential way.
From this one deduces the existence of a scheme $Q$, usually a subscheme of a Quot scheme, such that $X\times Q$ carries a vector $E$ such that all the  stable bundles in question occur among the fibres $E_q$. The moduli space $M$ would then  be quotient $M=Q/G$ for some appropriate reductive $G$ acting by "change of basis". At this point, to apply GIT,
one needs to know that stability in the abstract is related to to stability of the 
$G$-action. Note that $E$ typically won't descend i.e. $M$ need not be fine.
On the analytic side, one can form the quotient $N$ of the set of stable
complex structures $S$ on $V$ modulo gauge equivalence. Again stability comes into play
to guarantee that $N$ is reasonable. So then one gets a map of
spaces $Q^{an}\to S$ induced by $E$. This  should descend to a map $M^{an}\to N$. To check that this is an analytic equivalence, one would need a description of the local analytic structures on both sides. Fortunately one has  this this, see theorem 4.5.1 of [Huybrechts-Lehn]
and prop 6.4.3 of [DK]. Making this into a proof would take a lot of work of course.
