First, you should not dismiss the uniqueness of the connection too lightly. If you want to study a Riemannian metric per se, then you want to find invariants of it, things that are uniquely determined by the metric. Without the torsion-free assumption, there are many possible connections, and any properties derived from them will not be invariants of the metric. With the torsion-free assumption, the Levi-Civita connection is unique, so everything it implies is a property of the metric alone.

The next question is why not some other condition that might imply uniqueness of the connection? The torsion-free condition arises naturally enough to make it the natural one. The most important one is that, on a submanifold of Euclidean space, the flat connection on Euclidean space naturally induces a connection on the submanifold, and that connection is indeed torsion-free. Another property is that the Hessian of a function is always symmetric if and only if the connection is torsion-free.

Note also that when we study any mathematical object, we choose which properties we want to hold and that choice often depends on the depth and impact of the theory developed. Why do we assume that a Riemannian metric is symmetric? Why do we use an inner product metric and not a norm on the tangent space. When Anton says "it works", he is not talking specifically about parallel translation. He is referring to the entire rich subject of Riemannian geometry. People *have* studied connections that are not torsion-free, but so far the theory developed in that direction has not paid off nearly as much as Riemannian geometry has.