2
$\begingroup$

Question

Is Bregman divergence free of coordinates?

Although it is invariant w.r.t. which local affine coordinate you take, is it possible to prove that it does not change w.r.t. an arbitrary change of coordinates?

(I am reading "Information Geometry and Its Applications" (Amari, 2016).)

$\endgroup$
3
$\begingroup$

No. Away from a critical point, any function $\psi$ becomes linear in some coordinates. In such coordinates, the Bregman divergence $D_{\psi}$ of that function $\psi$, as defined in your book (p. 14 equation (1.44)), vanishes. So the Bregman divergence of a function, as defined in your book, is not coordinate invariant. It is, however, invariant under affine changes of coordinates.

On the other hand, consider the Bregman divergence to be defined not using affine coordinates and some function (as it is in your book), but instead to be defined as using a flat affine connection $\nabla$ with trivial monodromy and some differentiable function $\psi$, by $$D_{\psi,\nabla}(\xi,\eta)=\psi(\xi)-\psi(\eta)-d\psi(\xi)v$$ where $\exp_{\xi}v=\eta$ for the exponential map $\exp$ of $\nabla$. This Bregman divergence is coordinate independent, but dependent on not just the function: it depends also on the choice of the flat affine connection $\nabla$.

The dual connection is defined, in that book, using the Legendre transformation applied to $\psi$, so depends on $\psi$. Also, the Legendre transformation depends on the affine structure, which depends on the choice of the flat affine connection $\nabla$. So it doesn't make sense to pose a set of dual connections, without a function $\psi$. The notion of duality is not defined independently of $\psi$. But once you have $\psi$ and $\nabla$, you have the Bregman divergence $D_{\psi,\nabla}$ defined as above.

$\endgroup$
7
  • $\begingroup$ Thank you very much for your answer! Would you mind kindly clarifying some points for me? $\endgroup$
    – diadochos
    Oct 17 '17 at 14:23
  • $\begingroup$ (1) On the first line, did you mean by "its Bregman divergence" the convex function used for the construction? (So, on a coordinate where the convex function becomes linear, B-divergence vanishes, right?). (2) What is the role of "some function" mentioned in the last sentence? It seems to me that a canonical Bregman divergence can be derived from a set of dual connections. Is it for inducing a Riemannian metric (like a $C^2$ class convex function? $\endgroup$
    – diadochos
    Oct 17 '17 at 14:30
  • $\begingroup$ Ok, I added more details. $\endgroup$
    – Ben McKay
    Oct 17 '17 at 16:33
  • $\begingroup$ For some more information: in that book, the author finds local conditions on a pair of connections so that they are together carried by a local diffeomorphism into a dual pair for some convex function. So those results give a coordinate independent description of Bregman divergence, locally. The book doesn't make clear that they are only local results, but in that sense, they give a coordinate independent definition of a pair of affine connections arising via the construction of Bregman divergence. $\endgroup$
    – Ben McKay
    Oct 17 '17 at 16:37
  • $\begingroup$ May I ask you one last favor? Would you point out which part of the book you referred to when you said "the author finds local conditions on a pair of connections [...]" ? Anyway, thank you very much for giving me the clear answer! Along with the comments, I think I've learned much from it. I have also accepted your answer. $\endgroup$
    – diadochos
    Oct 18 '17 at 11:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.