You are thinking in terms of long geodesics, but your question is local. The metric is defined point by point. To know the metric tensor at a point $A,$ it is only necessary to know the geodesic distances from $A$ to all points in an arbitrarily small ball (germ) around $A.$ As to your second question, $S$ would need to be a fairly large set, as on $C^\infty$ manifolds one can alter the metric in a tiny ball (disk in dimension 2) without this being noticeable from far away.

You are thinking in terms of long geodesics, but your question is local. The metric is defined point by point. To know the metric tensor at a point $A,$ it is only necessary to know the geodesic distances from $A$ to all points in an arbitrarily small ball (germ) around $A.$ As to your second question, $S$ would need to be a fairly large set, as on $C^\infty$ manifolds one can alter the metric in a tiny ball (disk in dimension 2) without this being noticeable from far away. It may be that your set $S$ could be a finite set of triangulations, such that every point is in the interior of at least one simplex from at least one triangulation. So the idea, and I am not sure, is a Riemannian metric in the interior of a simplex determined by all pairwise distances between points on the boundary? If so, recovering the metric from such information is seriously nontrivial. But proving or disproving uniqueness is a smaller problem.