Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [information-geometry]

Information geometry is a branch of mathematics that applies the techniques of differential geometry to the field of probability theory. This is done by taking probability distributions for a statistical model as the points of a Riemannian manifold, forming a statistical manifold. The Fisher information metric provides the Riemannian metric.

5
votes
1answer
261 views

Approximation of Wasserstein distance between $p_\theta$ and $p_{\theta + d\theta}$

Given a parametric family of distributions $\{p_\theta\mid\theta \in \Theta\}$, one can show that under some regularity conditions, the following approximation is valid $$\operatorname{KL}(p_\theta\...
3
votes
1answer
115 views

Interpolation inequality related to the 5/3-Laplace operator

I'm having trouble with an estimate that would be helpful in information geometry. The background is the following. Suppose we have a smooth positive function $g:X \to \mathbb{R}^+$ where $X$ is a ...
6
votes
5answers
506 views

Reviews of Probability in High Dimension not by Van Handel

I'm completely in love with Ramon van Handel's lecture notes Probability in High Dimension and I would like to find more learning resources. Lecture notes or reviews would be ideal as anything in this ...
2
votes
0answers
102 views

Parametric distances on product spaces of measures

Disclaimer: Please excuse my loose language. I'm neither an expert in geometry nor probability. Please ask for clarification if something appears unclear or awkward to you. Let $X$ be a topological ...
3
votes
0answers
132 views

Distance measures that preserve Pythagoras' theorem but break the triangle inequality

In information geometry, we can think of the Kullback-Leibler divergence as being "something like a squared distance." The sense of this is that if we have three probability measures, $P$, $Q$ and $R$...
2
votes
1answer
130 views

Is Bregman divergence independent of coordinates?

Question Is Bregman divergence free of coordinates? Although it is invariant w.r.t. which local affine coordinate you take, is it possible to prove that it does not change w.r.t. an arbitrary change ...
3
votes
0answers
113 views

Covariance operator analogue for manifolds and respective measure manifolds

Assume $E$ is a connected riemannian manifold with geodesic metric space structure given by $d$ and $P$ is a probability measure over $E$ with Borel sigma-algebra given by this metric structure. Also ...
3
votes
0answers
164 views

Partial information decomposition for tangle machines

In (Williams and Beer, 2010), they define the partial information decomposition (PID) as a generalization of Shannon's Mutual Information for multiple information sources. Their key insight is that ...
2
votes
0answers
43 views

Minimizer of a class of SDEs

Setup Let $\mathscr{H}$ be a separable Hilbert space, $\mathcal{X}\triangleq \langle \Omega,\mathscr{F},\mathscr{F}_t,\mathbb{P}\rangle$ be a stochastic base and $X_t$ be an $H$-valued stochastic ...
2
votes
1answer
204 views

How much can KL divergence decrease by diluting the reference distribution

Let $\Omega$ be a countable set and $\mu,\nu\colon\Omega\to[0,1]$ be distributions on $\Omega$, that is we have $\sum_{x\in\Omega}\mu(x)=1$ and likewise for $\nu$. The Kullback-Leibler divergence of $\...
2
votes
1answer
178 views

(quasi)metric on Riemannian manifolds via Brownian Motion?

Given points $a$ and $b$ on a Riemannian manifold $M$, I would like a (quasi)metric that corresponds to some property of Brownian Motion from $a$ to $b$ (or rather, to $\epsilon$-ball $B = \{ x : |x - ...
4
votes
2answers
252 views

About optimization with Renyi divergence

Can someone link me to some pedagogic example of computing the Renyi divergence between two discrete/continuous distributions? Like examples where someone has been able to obtain a neat closed form or ...
1
vote
0answers
172 views

Relation between Aitchison Distance on a Simplex and Geodesic distance on the multinomial manifold [closed]

I am trying to understand the difference/relation between the Aitchison distance on a simplex $$\left[ \sum^D_{k=1} (\log{\frac{x_{ik}}{g(\mathbf{x}_i)}} - \log{\frac{x_{jk}}{g(\mathbf{x}_j)}})^2 \...
27
votes
0answers
10k views

Research situation in the field of Information Geometry

I am now doing an article survey on the field of information geometry started by S.Amari and Barndorff-Nielson. I want to know some research situation in this field. I have read (4) and parts of (3). ...