When can the metric be reconstructed (up to scaling) from knowing the conjugate points?

Let $M$ be a smooth manifold of dimension $\geq 2$. Let $g$ be a complete Riemannian metric on $M$. Let $C \subseteq M \times M$ be the set of pairs of $g$-conjugate points.

The set $C$ doesn't generally determine $g$, even up to scaling. For example hyperbolic space and Euclidean space both have no conjugate points.

Question: When can the metric $g$ be recovered (up to scaling) from knowledge of the pair $(M,C)$? Does this happen when $M$ is

1. A compact manifold?

2. A compact homogeneous space?

3. A compact symmetric space?

Do curvature conditions help? And in the noncompact case, can $g$ sometimes be recovered (up to scaling) from $C$?

Note: I've been a bit vague with the special cases I'm suggesting -- e.g. it's not clear if I'm asking whether for $M$ a compact homogeneous space, the conjugate points uniquely determine the metric among all metrics or just among homogeneous metrics. Ultimately I'd be interested in both questions, but I think the latter, more restricted question, is already interesting.

EDIT: As an example, I think I've almost convinced myself that for $n \geq 2$, the round metric on $\mathbb R \mathbb P^n$ can be reconstructed (up to scaling) from knowledge of the conjugate points.