There are several subtle aspects to these questions that I am sure have emerged for you since you first posted this question. The primary one is of course that probability spaces are assumed to have algebraic structures so that moments and other elements for statistical investigation of a random variable can be found -- or at least defined -- through direct analytic methods.

For example, the kth moment of a random variable with values in $(0,1)$ and probability distribution $f(x)$ is $\int_0^1 x^k f(x) dx.$ Note that you need to be able to multiply the values of the random variables to themselves and a real-valued function ($f$). Even multivariate statistics (which lacks multiplication) utilizes the vector-algebraic structure of $R^N$ to calculate moments etc.

All of this is lacking on a general manifold. Embedding the manifold into Euclidean space will not help, since you would typically get expected values, etc. living outside of the manifold, although it would be rather entertaining to tell someone that the expected location of the crash site of a satellite with decaying orbit is the center of the Earth.

Anyway, that's a long-winded introduction to point out that you should check out the following papers:

Pennec, X. "Probabilities and statistics on Riemannian manifolds: a geometric approach"
http://hal.inria.fr/inria-00071490/

Pennec, X. "Probabilities and statistics on Riemannian manifolds: Basic tools for geometric measurements." IEEE Workshop on Nonlinear Signal and Image Processing. Vol. 4. 1999.

Bochner in his text ${\it Harmonic\ Analysis\ and\ the\ Theory\ of\ Probability}$ describes stable distributions for general probability spaces, which is picked up for hyperbolic spaces in Getoor, R. K. "Infinitely divisible probabilities on the hyperbolic plane." Pacific Journal of Mathematics 11.4 (1961): 1287-1308.

valuesare manifolds, rather than a manifold on which a probability distribution is defined. – Joseph O'Rourke Oct 28 '11 at 18:25