I have a following scenario. there is a huge collection of data resulting from a random experiment $E$ (I do not say random variable yet, for reasons that you will need to explain in your answer). Let us call this a collection $D$. I have partitioned this randomly into $4$ equal parts $D_1,D_2,D_3,D_4$. The data points are nothing but a $d$-dimensional vectors, where $d$ is a very large number, $d$ higher than even the size of $D$. Now I want a low dimensional representation of this data but which preserves the geometry, that is inter-point distances of all the data points. For this I do two experiments $A$ and $B$.

Experiment $A$ :

Project all the data points of $D$ onto the span of all points in one segment, say $D_1$. This method preserves the interpoint distances of all points in $D_1$ exceedingly well, but does badly when you consider all points in $D$ rather than just $D_1$. The inter point distances of points from $D_2,D_3,D_4$ are not preserved satisfactorily, which is a very understandable thing. Similar is the case if you consider span of $D_2$ or $D_3$ or $D_4$.

Experiment $B$.

What I do is, I take one segment say $D_1$, and compute Eigen vectors $e_1,e_2,..e_k$ corresponding to top $k$ Eigen values (which contain the bulk of the energy) of the correlation matrix of data in $D_1$. Here $k<<n(D_1)$. Now I project all the data in $D$ to the span of $\{e_1,e_2,e_3,..e_k\}$, and surprisingly it preserves the interpoint disntances of all points in $D$ very well, (**all points in $D$ and not just in $D_1$**). This preserves the distances considerably better than in the case of old experiment $A$, where we project data in $D$ to the span of all points in $D_1$.

Is there any underlying mathematics that can explain this phenoemnon? What is the mathematics of the distribution of the random experiment that is generating this data. I want to know the strange facts about the underlying distribution of the random experiment that is generating this data.