# Questions tagged [information-geometry]

Information geometry is a branch of mathematics that applies the techniques of differential geometry to the field of probability theory. This is done by taking probability distributions for a statistical model as the points of a Riemannian manifold, forming a statistical manifold. The Fisher information metric provides the Riemannian metric.

30 questions
Filter by
Sorted by
Tagged with
188 views

### Which books should I read in order to be prepared to study information geometry?

At the moment, I am preparing my master's thesis (in statistics) and I intend to keep studying in order to pursue a doctoral degree. To be precise, I am mainly interested in studying Information ...
55 views

### Manifold structure of Gaussian mixtures

Fix $l$ a positive integer. Let $\mathcal{M}$ denote the set of Gaussian mixtures of the form $$\sum_{i=1}^l k_i \mu_i,$$ where $\mu_i$ is a non-degenerate Gaussian measure on $\mathbb{R}^k$ and ...
38 views

### Bounding the total variation metric between Gaussian mixtures

Let $\mathcal{P}(\mathbb{R}^d)$ the space of probability measures on $\mathbb{R}^d$ with total variation metric $\delta$, fix $k \in \mathbb{N}$, and let $\mathcal{P}'\subset \mathcal{P}(\mathbb{R}^d)$...
91 views

### Inequality on the Kullback-Leibler divergence

Let us define the arithmetic, geometric, and harmonic means of $x,y \in \mathbb{R}$ weighted by $\alpha =(\alpha_x,\alpha_y) \in [0,1]$, respectively as \begin{equation} a_\alpha(x,y) = \frac{\...
29 views

### Relation between the covariance of a random variable taking values in an embedded submanifold and the covariance matrix in the ambient Euclidean space

Let $M^m \subset \mathbb{R}^d, m < d$ be an $m$-dimensional embedded submanifold. Let $X: \Omega \to M^m$ be a manifold valued random variable. Then we've apparently two different notions of ...
185 views

### Pythagorean theorems for other distances

Question The usual projection in $\mathbb{R}^n$ on a subspace can be defined as the point that minimizes the squared distance to the subspace. I'll call the Pythagorean theorem the easy fact that, ...
93 views

### Connecting Wasserstein distance with mutual information?

Suppose I have Markov chains: $$X \rightarrow f(X) \rightarrow g(X)$$ $$Y \rightarrow f(Y) \rightarrow g(Y)$$ where it is known that minimizing the $\mathbb{E}(g(X)) - \mathbb{E}(g(Y))$ minimizes the ...
91 views

478 views

### Relation between information geometry and geometric deep learning

Disclaimer: This is a cross-post from a very similar question on math.SE. I allowed myself to post it here after reading this meta post about cross-posting between mathoverflow and math.SE, I did ...
115 views

121 views

122 views

### Interpolation inequality related to the 5/3-Laplace operator

I'm having trouble with an estimate that would be helpful in information geometry. The background is the following. Suppose we have a smooth positive function $g:X \to \mathbb{R}^+$ where $X$ is a ...
1k views

### Reviews of Probability in High Dimension not by Van Handel

I'm completely in love with Ramon van Handel's lecture notes Probability in High Dimension and I would like to find more learning resources. Lecture notes or reviews would be ideal as anything in this ...
131 views

### Parametric distances on product spaces of measures

Disclaimer: Please excuse my loose language. I'm neither an expert in geometry nor probability. Please ask for clarification if something appears unclear or awkward to you. Let $X$ be a topological ...
151 views

### Distance measures that preserve Pythagoras' theorem but break the triangle inequality

In information geometry, we can think of the Kullback-Leibler divergence as being "something like a squared distance." The sense of this is that if we have three probability measures, $P$, $Q$ and $R$...
153 views

### Is Bregman divergence independent of coordinates?

Question Is Bregman divergence free of coordinates? Although it is invariant w.r.t. which local affine coordinate you take, is it possible to prove that it does not change w.r.t. an arbitrary change ...
138 views

### Covariance operator analogue for manifolds and respective measure manifolds

Assume $E$ is a connected riemannian manifold with geodesic metric space structure given by $d$ and $P$ is a probability measure over $E$ with Borel sigma-algebra given by this metric structure. Also ...
197 views

### Partial information decomposition for tangle machines

In (Williams and Beer, 2010), they define the partial information decomposition (PID) as a generalization of Shannon's Mutual Information for multiple information sources. Their key insight is that ...
43 views

### Minimizer of a class of SDEs

Setup Let $\mathscr{H}$ be a separable Hilbert space, $\mathcal{X}\triangleq \langle \Omega,\mathscr{F},\mathscr{F}_t,\mathbb{P}\rangle$ be a stochastic base and $X_t$ be an $H$-valued stochastic ...
222 views

342 views

### About optimization with Renyi divergence

Can someone link me to some pedagogic example of computing the Renyi divergence between two discrete/continuous distributions? Like examples where someone has been able to obtain a neat closed form or ...
I am trying to understand the difference/relation between the Aitchison distance on a simplex \left[ \sum^D_{k=1} (\log{\frac{x_{ik}}{g(\mathbf{x}_i)}} - \log{\frac{x_{jk}}{g(\mathbf{x}_j)}})^2 \...