The other answers give good insight. Here's another perspective.
Since the Levi-Civita connection is the unique metric and torsion-free connection, to motivate its use we need to convince ourselves that both of these properties are desirable. I'll note that there is sometimes value in considering non-metric connections, but in the question you addressed why using metric connections make sense for studying geometry. So I guess the real issue is to tackle torsion-free-ness.
In order to address this, the first thing to do is try to understand what torsion really is anyway. There is another question on Mathoverflow about torsion with some great answers, but let me try to draw some pictures. We'll start with the standard picture of the curvature tensor (for a torsion-free connection).
The idea is that we have three vectors $X$, $Y$ and $Z$. Starting at a point $P$ in our space, we use our connection to parallel transport $Z$ an infinitesimal amount along a geodesic in the $X$ direction and then along a geodesic the $Y$ direction. We then parallel transport $Z$ in an infinitesimal amount in the $Y$ direction and then the $X$ direction. The curvature measures the difference between these two parallel transports. In the formula, the Lie bracket term is there to make sure that everything is nice and tensorial.
What changes if the torsion is non-zero?
In this case, if we parallel transport in the $X$ then $Y$ directions versus the $Y$ then $X$ directions, the infinitesimal square doesn't close up, and what remains is the torsion. Again, in the formula there is a Lie bracket term to make everything nice and tensorial.
So why do we want a torsion free connection?
In my opinion, torsion is complicated invariant and is somewhat hard to understand. For curvature, there is a very clear picture of what it means for a space to have positive versus negative curvature (infinitesimal planes coming together versus spreading apart). As such, it's possible to formulate all sorts of theorems in terms of curvature assumptions. On the other hand, torsion is this awkward vector that you get when you compute multiple derivatives. It's not really meaningful for it to be "positive" or "negative," and so it doesn't affect the analysis in predictable ways. As such, life is often a lot easier when it's not around, and is what makes the Levi-Civita connection so useful.
I should add that there are times where considering connections with torsion makes sense. For instance, on a Lie group it is possible to construct a curvature-free connection whose torsion encodes the Lie algebra. This is a very useful connection, but from an analytic perspective, it's not so clear geometrically how the respective torsions of $SO(3)$ versus the Heisenberg group (for instance) give rise to their very different geometries. Another example is in non-Kahler complex geometry, where we can study holomorphic, complex, metric connections, which must have non-zero torsion. But again, even though the torsion is present and necessary, it's often hard to really use it in a meaningful way.