A short exact sequence $$0\to A\to B\to C\to 0$$ tells you that to understand $B$ you often can split the work in three parts: understand $A$ first, then understand $C$, then finally try to glue your understanding of $A$ and $C$ into understanding of $B$ itself.

They show up all over the place because this strategy is very often successful.

*P.S.* Sometimes one proceeds differently. For example, you may be interested inknowing something about $A$, but it is difficult, so you find a larger $B$, which is hopefully easier to study, but then the information you got is not about $A$ but about $B$, so $C$ measures the difference. Similarly, one is often interested in $C$, and it is convenient to describe it as the quotient of a simpler object $B$: but then $B$ and $C$ are not the same thing, of course, and you need to study their difference, which is encoded in the object $A$.