As a follow up to me other question, what can be said about unstable vector bundles? I know this is rather open ended, but what sorts of horrible things does having a subbundle of strictly greater slope imply?
Well, I don't know about horrible. There's a lot you can say that's good! I'll start rambling and see where I end up.
I'm going to pretend you said principal GL(n)-bundle instead of rank n vector bundle. Same thing, really, since we have the standard representation.
The collection Bun(n,C) of all principal GL(n) bundles P on a smooth curve C is a very nice geometric object: it's an Artin stack. It's not connected; the different components are labelled by topological data, like the Chern class. The tangent "space" (complex, really) to Bun(n,C) at a point P is naturally the derived global sections RGamma(C,ad(P)), where ad(P) is the associated bundle with fiber the adjoint representation of GL(n). The zero-th cohomology gives the infinitesimal automorphisms and the 1st cohomology gives the deformations. So the stabilizer group of any point V in Bun(n,C) is finite-dimensional, and the dimension of the stack is n(g-1) (by Riemann-Roch). Bun(n,C) is smooth, and unobstructed, thanks to the vanishing of H^2(C,ad(P)).
Bun(n,C) has a very nice stratification, too. It's an increasing union of quotient stacks [A/G] of projective varieties by finite-dimensional groups. Roughly, A is the stack of pairs (P,t), where t is a trivialization of P in an infinitesimal neighborhood of some point in C. Make the neighborhood large enough, i.e., r-th order, and you can kill off all the automorphisms of P. Unfortunately, except for n=1, there is no uniform bound on r that works for all bundles. So, Bun(n,C) isn't a finite type quotient stack.
You can also realize Bun(n,C) (homotopically) as the infinite type quotient stack of U(n)- connections modulo complexified gauge transformations. That's what Atiyah & Bott do in their paper "The Yang-Mills Equations on Riemann Surfaces". (They also have a nice discussion of slope-stability and the stratification.)
The top component of the stratification (those bundles where the stabilizer group is as small as possible) is the stack of (semi-)stable vector bundles. If you take the coarse moduli space of this substack, you get the usual moduli space of stable bundles.
In summary: If you drop the stability conditions, you get a lot more geometry with a similar flavor, and without the random bits of weirdness that crop up in the theory of moduli spaces. (e.g., the stack always carries a universal bundle, you don't need the rank and the chern class to be coprime.)
OK, I'll stop evangelizing now.
Reider's proof of the Fujita conjecture that on a smooth surface, K+4A is very ample whenever A is ample, relies fundamentally on the Bogomolov instability theorem. I.e. It is precisely the Bogomolov instability of the vector bundle produced by Serre's construction associated to a possible base point of a linear series K+L, under certain conditions on the chern classes, that allows one to conclude the result. This technique seems to have played a major role in the study of linear series since that time. See Igor Reider's paper in the Annals of Math, (1988), or the article of Lazarsfeld in volume 3 of the IAS Park City series, Complex algebraic geometry.
If you prefer to phrase it as bad behavior related to instability, I guess you could, since this shows that the presence of something bad, namely a base point, implies instability, which forces something else (the presence of certain curves though the base point on the surface) which if not bad, is at least rather special. And this lets you classify all cases where the original bad behavior occurs. So certain specific bundles can only be unstable in rather unusual ways.
Another reason behind the notion of stability is the "jumping phenomenon." Namely, you can construct a family of vector bundles parametrized by the disk where all the fibers apart from the origin are mutually isomorphic, but not isomorphic to the fiber at the origin. Concretely, you can realize this by scaling an extension class. In the Atiyah-Bott-Kempf-Ness picture, this corresponds to non-closed orbits of the complex gauge group. When the base manifold has dimension greater than one, the pieces in the Harder-Narasimahan filtration may not be subbundles, so the limiting object is in general a torsion-free sheaf, and not necessarily a bundle.