Ah, I forgot to specify that. Assume that it's the spectral norm (the 2-norm). However, the Frobenius norm is also fine since $\|A\|_2 \le \|A\|_F\le \sqrt{d} \|A\|_2$ when $d$ is the ambient dimension.

I think there is a misunderstanding. My question is on whether the error term of the 1st order Taylor expansion of $\exp_x$ can be realized as $(t^2/2) Q_x(w)$ for some appropriate $w \in T_x M$ with $\|w\| \le 1$. I don't think that this follows directly from Taylor's theorem? The best shot seems to be applying Taylor's theorem coordinate-wise to $t \mapsto \exp_x(tv)$ but that won't imply the desired result directly. Also, $J_x: T_x M \rightarrow \mathbb{R}^n$ is a required notation if we regard the tangent space as not a priori embedded in $\mathbb{R}^n$.

I think I have a resolution to the question asked, by applying Taylor's theorem coordinate-wise to the exponential map $t \mapsto \exp_x(tv)$ and bounding the error term crudely. This seems to yield that the error term is bounded by $Bt^2$ where $B = \frac n2 \sup_{w \in TM} \| \nabla_w w \| $ and $n$ is the ambient dimension. I'll clarify after I get some sleep.

ah, someone added a typo while editing. I'm going to delete that $>$.

I was being imprecise, but I think it's certainly injective for measures induced by continuous probability density functions.

Also, could you tell me if the converse statement stated in the first paragraph is true?

Those are delightful. May I know the references or Google-able keywords for those facts?

oh, I thought you meant having five subsets, rather than subsets of size five.