First, you should not dismiss the uniqueness of the connection too lightly. If you want to study a Riemannian metric per se, then you want to find invariants of it, things that are uniquely determined by the metric. Without the torsion-free assumption, there are many possible connections, and any properties derived from them will not be invariants of the metric. With the torsion-free assumption, the Levi-Civita connection is unique, so everything it implies is a property of the metric alone.

The next question is why not some other condition that might imply uniqueness of the connection? The torsion-free condition arises naturally enough to make it the natural one. The most important one is that, on a submanifold of Euclidean space, the flat connection on Euclidean space naturally induces a connection on the submanifold, and that connection is indeed torsion-free. Another property is that the Hessian of a function is always symmetric if and only if the connection is torsion-free.