# What structure is needed to define a Gaussian distribution on a given space?

In most textbooks, the normal distribution is defined on $\mathbb{R}^n$ by specifying its probability density function. This works perfectly well, but it isn't really amenable to generalisation.

I'm wondering what the minimal structure is that one must have on a given space $S$ before one can define an analogue of the normal distribution. If $S$ has a Riemannian metric defined on it, then one can define a Brownian motion on $S$ using the Laplace-Beltrami operator. The family of normal distributions on $S$ could then be defined as the one-dimensional marginals of the Brownian motion.

Alternatively, the normal distribution could be characterised as the distribution that maximises entropy when its mean and variance are known. This characterisation only seems to require a notion of "mean" (which suggests that $S$ must be a metric space).

Is there a more general construction of the normal distribution?

-
how about regarding a / the solution of the associated heat equation on that space / manifold? would that correspond to a generalized gaussian? – Suvrit Mar 27 '11 at 22:07
Yes, that's more or less equivalent to constructing a Brownian motion on the space. You need some generalisation of the Laplace operator. – Simon Lyons Mar 27 '11 at 22:13
"In most textbooks, the normal distribution is defined on $\mathbb{R}^n$ by specifying its probability density function." Really? Specifically which books? The most usual definition in my experience is this: A random variable with values in $\mathbb{R}^n$ is normal iff its dot product with every constant (i.e. non-random) vector has a 1-dimensional normal distribution. How do the books you have in mind deal with the case where the variance $E((X-\mu)(X-\mu)^T)$ is singular? The vector of residuals (as opposed to errors) in linear regression is such a case. – Michael Hardy Mar 27 '11 at 22:40
One also see this: If $A$ is an $m\times n$ matrix, $b$ is an $m\times 1$ column vector, and $\mathbb{Z} = (Z_1,\dots,Z_n)^T$ has independent normally distributed components each with a $N(0,1)$ distribution, then $AZ$ has an $m$-dimensional normal distribution. In that case the expected value is $b$ and the variance is the $m\times m$ matrix $AA^T$ (which of course in some cases is singular). – Michael Hardy Mar 27 '11 at 22:51
PS: I don't mean to suggest that defining them via the density can't or shouldn't be done. It would have to be a density with respect to a measure on a subspace of dimension in some cases lower than the dimension of the ambient space. Somehow it seems as if that could get messy, but I've never thought it through. – Michael Hardy Mar 27 '11 at 22:58

Let me assume that you seek for the generalization of Gaussian distribution in order to generalize the Brownian motion.

As far as I know, regarding the heat kernel as the generalization of the Gaussian distribution has long been adopted in many literatures. It comes from the following observation.

In $\mathbb {R}^1$, the following notions coincide:

(1) Gaussian distribution $N(x,t)\sim f(t,x,y)=\frac {1}{\sqrt{2\pi t}}e^{\frac{-(y-x)^2}{2t}}$,

(2) transition function $p(t,x,y)$ of the Brownian motion $B_t$,

(3) (heat kernel) fundamental solution $k_t(x,y)$ of the heat equation $\partial_t k=\Delta_y k$, with initial data $\delta_x$.

Thus, on manifolds, one way to define the Brownian motion is to construct a Markov process on the manifold whose transition function is exactly the heat kernel (let's identify the heat kernel with the Gaussian distribution in this setting). Since we always have the Laplacian-Beltrami $\Delta$ on a manifold, it is justifiable to talk about the heat equation and thus the heat kernel, and the Brownian motion in this sense is known to exist for a large class of manifolds.

But on metric spaces, we no longer have the Laplacian-Beltrami. So, in order to talk about heat kernel/Gaussian distribution, we need to generalize the notion of Laplacian-Beltrami. The key concept on this line the so-called Dirichlet form. A Dirichlet form on metric measure space $(X,d,\mu)$ a closed symmetric form $(\cdot,\cdot)$ defined on $L^2(X,\mu)$. It should further satisfy a couple of conditions so that it behaves like its prototype $(f,g)=\int_{M} {\nabla f\cdot \nabla g dx}$ on a manifold $M$. Notice that $(f,g)=(-\Delta f,g)_{L^2(M)}$ on manifolds, in the general case, one obtains the desired "Laplacian" by the same formula. Therefore, every Dirichlet form corresponds to a "Laplacian" and thus a Gaussian distribution (and thus a Brownian motion). What's more, a reasonable Dirichlet form always exists provided the space is suitably good.

In sum, if the space you are considering have both metric and measure structures, then the theory of Dirichlet form may provide you some satisfactory results regarding construction and properties of the Guassian distribution (and thus the Brownian motion). Roughly speaking, if we don't have a presumed measure, we may not be able to construct a reasonable probability space; if we don't have a metric, it would be hard to measure the regularity and decay of the Gaussian distribution. So metric measure structure might be the minimal structure for reasonable construction of Gaussian distribution.

Some reference books could be found in the above link. This paper by Sturm may allow you to have a glance at the whole picture. I am not an expert in this field. I apologize in advance for any mistake and naivety.

-
Great answer, thanks – Simon Lyons Mar 28 '11 at 14:40
The requirement to have a reference measure $\mu$ can be troublesome though. In particular it prevents you from using this machinery to construct Gaussian measures on Banach spaces in the senses of the answers of Tom LaGatta and Mark Meckes; by analogy with the finite dimensional case, the natural reference measure to use would be Lebesgue measure, but in infinite dimensions there is no such thing, – Nate Eldredge Mar 28 '11 at 15:46

There is a standard definition for a Gaussian distribution on a Banach space (see e.g. Probability in Banach Spaces by Ledoux and Talagrand). A random vector $X$ with values in a Banach space $B$ is called Gaussian if it is Radon (a regularity condition which, for simplicity, I won't define here), for every $f \in B^*$, the $\mathbb{R}$-valued random variable $f(X)$ is Gaussian. This is a natural infinite-dimensional extension of the characterization suggested in Michael Hardy's first comment.

An important point here is that in infinite dimensions this gives a way to talk about a Gaussian distribution, but there is no longer any notion of the standard Gaussian distribution.

-

Let $X$ be a Banach space with its Borel $\sigma$-algebra $\mathcal B(X)$, and let $\mathbb P$ be a Radon measure on $X$. The continuous linear functionals on $X$ are measurable functions, hence random variables. Let's assume that for all $f \in X^*$, $$\mathbb E|f|^2 = \int_X |f(x)|^2 \ \mathrm{d}\mathbb P(x) < \infty.$$ This assumption implies a powerful structure theorem (cf. Theorem III.2.1 of [1]):

Theorem. There exists an element $\mu \in X$ (the mean element) and a continuous, symmetric operator $K : X^* \to X$ (the covariance operator) such that for all $f, g \in X^*$, $$f(\mu) = \mathbb E f \qquad \mathrm{and} \qquad f(Kg) = \mathbb E(fg) - f(\mu)g(\mu).$$

Furthermore, the support of the measure $\mathbb P$ is contained in the affine subspace spanned by the mean and covariance operator: $$\operatorname{supp} \mathbb P \subseteq \mu + \overline{KX^*},$$ and this space is separable.

As Mark Meckes said, we say that the measure $\mathbb P$ is Gaussian if each $f \in X^*$ is a real-valued Gaussian random variable (hence with mean $f(\mu)$ and variance $f(Kf)$. In this case, the support is the full affine subspace $\mu + \overline{KX^*}$. Furthermore, the measure is completely characterized by its characteristic functional $\varphi : X^* \to \mathbb R$: $$\varphi(f) = \int_X \mathrm e^{\mathrm i f(x)} \ \mathrm d\mathbb P(x) = \mathrm e^{\mathrm i f(\mu) - f(Kf)/2}.$$

For more on this topic, I suggest the reference:

[1] Probability Distributions on Banach Spaces by Vakhania, Tarieladze and Chobanyan.

-
Thanks for your answer Tom. I don't have much intuition about Banach spaces and their duals. Is that a restrictive assumption? Can one always find such a measure $\mathbb{P}$? – Simon Lyons Mar 28 '11 at 23:33
Well, a continuous Gaussian process on a compact parameter set $T$ (like the Wiener process on $[0,1]$) is exactly the realization of a Gaussian measure on the Banach space of continuous functions $C(T)$. Here, the covariance operator corresponds to the integral operator of the covariance function. Any positive-definite function serves as the covariance function to some measure, though there is a special class of them which corresponds to Gaussian processes, and a finer subclass which corresponds to continuous Gaussian processes. – Tom LaGatta Mar 29 '11 at 5:44
So the short answer is: it's not a restrictive assumption, in that there are LOTS of covariance functions for continuous Gaussian measures, though not every positive-definite function gives rise to one. – Tom LaGatta Mar 29 '11 at 5:45

One general construction can be found in Revuz and Yor "Continuous Martingales and Brownian Motion" for instance:

proposition (1.3)

Let $H$ be a separable real Hilbert space. There exist a probability space $\left( \Omega, \mathcal{F}, \mathbb{P} \right)$ and a family $X(h)$, $h \in H$ of random variables on the space such that

i) the map $h \rightarrow X(h)$ is linear

ii) for each $h$ the random variable $X(h)$ is Gaussian centered and $\mathbb{E} [ X(h)^2] = ||h||_{H}^{2}$

Nualarts book "Malliavian Calculus" also starts with the notion of isonormal Gaussian process which is general as well as Adler's books on Gaussian processes.

Alternatively you could look at one of T. Hida's books on White noise analysis for a construction based on the bochner-milnos theorem and Nuclear spaces.

Sorry none of these have a geometric perspective that I am aware of...

-
Thanks for your comment. The construction in Nualart that you mentioned is for a set of one-dimensional Gaussians indexed by a Hilbert space. I'm really looking for analogues of the normal distribution that take values in a space $S$. – Simon Lyons Mar 27 '11 at 19:57
sorry is it appropriate to remove this as answer then? One last thing I can add is the construction of "brownian motion" in the free potability setting where you are working over von neumann algebras of operators as in section 1.1 of iecn.u-nancy.fr/~nourdin/4th-moment-Wigner-KNPS.pdf. Just in case you were unaware of an example of a type of Brownian motion taking values in a non commutative space. – jzadeh Mar 27 '11 at 20:15
I think this is the correct generalization of normal distributions to the infinite dimensional setting. The process $X$ does not itslef lie in the Hilbert space though (see my old answer mathoverflow.net/questions/19020/brownian-motion-and-spheres/…;. – George Lowther Mar 27 '11 at 20:57