Whenever I have seen unique factorization discussed, it is always with respect to the solution of diophantine equations; the equations are solved by splitting the equation into linear functions over a ring and then invoking unique factorization. But the discussions always give the impression that the failure of unique factorization causes all sorts of problems. Apart from the applications to diophantine equations, why is unique factorization such a desirable property?
I think a glance at any elementary number theory textbook will show you numerous topics which rely on uniqueness of factorization. The formulas for the number of divisors, the sum of the divisors, the Euler phifunction, all depend on unique factorization. Going up a level, the Euler product for the Riemann zeta function depends on unique factorization.

1$\begingroup$ Your last statement seems dubious to me: Dedekind zeta functions also have Euler products. The key property for this is unique factorization of ideals, i.e., that the ring of integers of a number field is a Dedekind domain. $\endgroup$ – Pete L. Clark Jun 1 '10 at 5:16

1$\begingroup$ @Pete, I would say unique factorization into ideals is the solution of the problem caused by the failure of unique factorization into elements; I interpreted the question to be about problems caused by the failure of unique factorization into elements. If OP wants to know why unique factorization into ideals is such a desirable property, I'd nominate the Dedekind zeta functions as part of the answer. $\endgroup$ – Gerry Myerson Jun 1 '10 at 5:54
An integral domain which satisfies the ascending chain condition on principal ideals (ACCP) is a UFD iff every irreducible element is prime. This is a very useful property  in Diophantine equations and otherwise  and is so natural that it takes some training to remember that in an arbitrary domain irreducibles need not have this stronger property.
There are other characterizations of UFDs which show that a nonUFD lacks certain nice properties. For instance, a Noetherian domain is a UFD iff every height one prime ideal is principal. Applied to the polynomial ring $k[t_1,\ldots,t_n]$, this shows that every irreducible codimension one subvariety is cut out by a single equation.
Cartier and Weil divisors coincide on a Noetherian integral scheme iff it is locally factorial. Combined with the fact that any regular local ring is factorial, this gives the equivalence between divisors and line bundles on a nonsingular variety. Turned around, this shows that there is "something wrong" with a domain which is not at least locally factorial: it has singularities.
Also, a domain satisfying (ACCP) is a UFD iff every two elements have a greatest common divisor (iff every two elements have a least common multiple). Thus in a nonUFD one cannot necessarily reduce an element of the fraction field to lowest terms.

$\begingroup$ I think the last paragraph here should have gone first: it is a very concrete illustration of something weird that can happen. In a Dedekind domain the property of being able to write every ratio of nonzero elements in its fraction field in "lowest terms" is in fact equivalent to it being a UFD. $\endgroup$ – KConrad Feb 27 '16 at 20:01