2
$\begingroup$

Let $\mathcal{P}:=\mathcal{P}(\mathcal{X})$ be the manifold of all (strictly positive) probability vectors (distributions) on $\mathcal{X}=\{x_0,\dots,x_n\}$,

i.e., each $p=(p(x_0),\dots,p(x_n))\in \mathcal{P}$ is such that $p(x_i)>0$ for all $i$ and $\sum_{i}p(x_i)=1$ and can be thought of a point in $\mathbb{R}^{n+1}$, i.e., $\mathcal{P}$ is an $n$-dimensional manifold.

Let $\mathcal{P}=\{p_{\xi}\}$, where $\xi=(\xi_1,\dots,\xi_n)$, be a (global) coordinate system. For example, $\mathcal{P}$ can be parametrized by $\mathcal{P} = \{p_{\xi}\}$, $\xi = (\xi_1,\dots,\xi_n)\in \Xi,$ where \begin{eqnarray*} \Xi = \{(\xi_1,\dots,\xi_n) : \xi_i>0, \sum_{i=1}^n\xi_i < 1\}, \end{eqnarray*} with \begin{equation} p_{\xi}(x_i) = \begin{cases} \xi_i & \text{for } i = 1,\dots,n \\ 1-\sum_{i=1}^n\xi_i &\text{for } i = 0. \end{cases} \end{equation}

A Riemannian metric $G(\xi) = [g_{i,j}(\xi)]$ is defined on $\mathcal{P}$, where \begin{eqnarray} g_{i,j}(\xi) & = & \sum_x \frac{\partial}{\partial\xi_i} (p_{\xi}(x))~ \frac{\partial}{\partial\xi_j}(\log p_{\xi}(x)), \end{eqnarray} where the $\log$ is taken coordinatewise.

An affine connection $\nabla$ is defined on $\mathcal{P}$, given by the Christoffel symbols \begin{eqnarray} \Gamma_{ij}^k({\xi}) & = & \sum_x \frac{\partial}{\partial\xi_k}(p_{\xi}(x))~\frac{\partial}{\partial\xi_i}\left(\frac{\partial}{\partial\xi_j}\log p_{\xi}(x)\right). \end{eqnarray} Suppose that $\gamma_t$ is a $\nabla$-geodesic on $\mathcal{P}$. From THIS article of Amari in Ann. of Statistics, I came to know that the geodesic equation (for this connection) is given by $\ddot l_t+i_t=0$, where $l_t(x)=\log\gamma_t(x)$, and $i_t=\sum_x \dot\gamma_t(x)\dot l_t(x)=0$. However, the paper doesn't seem to give any explanation how it is obtained (See Appendix of the paper. $\alpha=1$ corresponds to my question). If anyone can help me derive this geodesic equation, it would be of great help. Thank you.

PS: I asked this question some days ago in math.stackexchange where I couldn't get any response. Here's a link to the question.

$\endgroup$
  • 1
    $\begingroup$ The answer depends on which is your favourite definition of a geodesic. A possibility would be to take that equation as the definition of a geodesic. $\endgroup$ – Paolo Ghiggini May 19 '14 at 16:36
  • $\begingroup$ $\nabla_{\dot\gamma_t}\dot\gamma_t=0$ is the most common geodesic equation, I presume. I am wondering which choice of geodesic definition would give raise to $\ddot l_t+i_t=0$. $\endgroup$ – Ashok May 19 '14 at 16:46
  • $\begingroup$ Your link of Amari's article directs to your post in math.stackexchange. I've tried to read an article of Amari, which I guess to be the same one, but I can't achieve a consistent interpretation of his notions at that time. His setting is really unusual to differential geometers like me. Wish to see an answer to this post. $\endgroup$ – Chih-Wei Chen May 20 '14 at 5:21
  • $\begingroup$ @Chih-WeiChen: I have added the reference. $\endgroup$ – Ashok May 20 '14 at 6:52
7
$\begingroup$

Let me first re-write your notations as little bit to make it easier for me.

Let $\xi\in \Xi \subset\mathbb{R}^n$. Define the functions $\pi_k:\Xi \to\mathbb{R}$ by

$$ \pi_k(\xi) = \begin{cases} \xi_k & k \in \{1, \dots, n\} \\ 1 - \sum_{1}^n \xi_i & k = 0 \end{cases} $$

which to me is a more natural way to write your function $p_\xi: \{0, \ldots, n\} \to \mathbb{R}$.

Your metric is defined to be

$$ \mathbf{g} = \sum_{k = 0}^n \frac{1}{\pi_k} ~\mathrm{d}\pi_k \otimes \mathrm{d}\pi_k = \sum_{k = 0}^n \mathrm{d}\pi_k \otimes \mathrm{d} \log \pi_k $$

What you write as the Christoffel symbols is not really the Christoffel symbol, it is the Christoffel symbol with one index lowered. Note that the Christoffel symbol for the Levi-Civita connection is given by

$$ g_{k\ell} \Gamma^{\ell}_{ij} = \frac12 \left( \partial_i g_{kj} + \partial_j g_{ki} - \partial_k g_{ij}\right) $$

one check indeed

$$ g_{k\ell} \Gamma^{\ell}_{ij} = \sum_{m = 0}^n \partial_k \pi_m \partial^2_{ij} \log\pi_m = - \sum_{m = 0}^n \frac{1}{\pi_m^2} \partial_k \pi_m \partial_i \pi_m \partial_j \pi_m $$

where in the second equality we used that $\pi_m$ are linear in the coordinates and hence the Hessian vanishes.

Let $\gamma:\mathbb{R}\to \Xi$ be a geodesic. The geodesic equation in coordinates can be written as

$$ g_{k\ell}\circ\gamma~ \ddot{\gamma}^\ell + g_{k\ell}\circ\gamma~ \Gamma^\ell_{ij}\circ\gamma ~\dot{\gamma}^i \dot{\gamma}^j = 0$$

plugging in the definitions we have

$$ \sum_{m = 0}^n (\partial_k \pi_m)\circ\gamma \left[ (\partial_\ell \log \pi_m) \circ\gamma ~\ddot{\gamma}^{\ell} + (\partial^2_{ij} \log\pi_m)\circ\gamma~ \dot{\gamma}^i \dot{\gamma}^j \right] = 0 $$

which we simplify as

$$ \sum_{m = 0}^n (\partial_k \pi_m) \circ \gamma \frac{d^2}{dt^2}( \log\circ\pi_m\circ \gamma) = 0 $$

Now, $\partial_k \pi_m = \delta_{km}$ if $m > 0$ and $-1$ otherwise. So we have

$$ \frac{d^2}{dt^2}( \log\circ\pi_k\circ \gamma) - \frac{d^2}{dt^2}(\log\circ\pi_0 \circ \gamma) = 0 \tag{1}$$


This is almost what you want. To finish, we observe the following:

$$ \sum_{m = 0}^n \pi_m\circ \gamma = 1 \implies \sum_{m = 0}^n \frac{d^k}{dt^k} \pi_m\circ\gamma = 0, k \geq 1 \tag{2}$$

Observe that

$$ \frac{d^2}{dt^2} \log \circ\pi_m \circ \gamma = \frac{1}{\pi_m\circ\gamma} \frac{d^2}{dt^2} \pi_m\circ\gamma - \frac{1}{(\pi_m\circ\gamma)^2} (\frac{d}{dt} \pi_m\circ\gamma)^2 $$

This says that

$$ \sum_{m = 0}^n \pi_m\circ\gamma \frac{d^2}{dt^2} \log\circ\pi_m\circ\gamma = - \sum_{m = 0}^n \frac{1}{(\pi_m\circ\gamma)} (\frac{d}{dt} \pi_m\circ\gamma)^2 \tag{3}$$

Applying (3) to (1) gives

$$ \sum_{m = 0}^n \pi_m\circ\gamma \frac{d^2}{dt^2} \log\circ\pi_0 \circ\gamma = - \sum_{m = 0}^n \frac{1}{(\pi_m\circ\gamma)} (\frac{d}{dt} \pi_m\circ\gamma)^2 $$

so from (2), and (1) again, we get, for every $k \in \{0,\dots, n\}$,

$$ \underbrace{\frac{d^2}{dt^2} \log\circ\pi_k \circ\gamma}_{\ddot{l}_t(x_k)} + \sum_{m = 0}^n \underbrace{\frac{d}{dt}(\pi_m\circ\gamma)}_{\dot{\gamma}_t(x_m)} \underbrace{\frac{d}{dt} \log\circ\pi_m\circ\gamma}_{\dot{l}_t(x_m)} = 0 $$

which is exactly the equation claimed in the question.


I should remark that part of the above answer is reconstructed in differential geometry language from the linked paper. In some ways the original derivation is a bit slicker, especially when it comes to the derivative of what I wrote as equation (3).

Addendum to address this final paragraph:

What you are treating is really the Fisher information metric. Consider the set $\Xi$ which is the interior of the simplex. If you look at the functions $\eta_i$ such that $(\eta_i)^2 = \xi_i$, the simplex is defined as $\sum (\eta_i)^2 = 1$, with $\eta_i \in (0,1)$. Notice that this can be interpreted as a quadrant of the unit sphere with $n$ dimensions (so $i$ runs from $0$ to $n$).

The metric you wrote down in $\xi$ coordinates turns to to be exactly the pull back metric of the standard unit sphere (up to some constant multiples) to $\Xi$ using the mapping $(\eta_i)^2 = \xi_i$.

The geodesics on the unit sphere are the great circles and are thus the intersections of the sphere with hyperplanes. Let $\zeta \in \mathbb{R}^{n+1}$ be a fixed vector, there is a geodesic in $\eta$ coordinates satisfying $\sum \zeta_i \eta_i = 0$. We can choose the affine parametrisation $\eta_i(t)$ to be such that $\sum (\frac{d}{dt}\eta_i(t))^2 = E$ is constant. It is well-known (since Newton) that in this parametrisation the acceleration of $\eta$ is in the direction of $\eta$ itself, with the proportionality factor $E$: that is

$$ \ddot{\eta} + E \eta = 0. $$

Doing the reverse change of variables you arrive back to the formula for the expression of the geodesics in the $\xi$ coordinates.

The fact that $\dot{\ell}$ in the language of the paper is a tangent vector, is exactly the statement that in $\eta$ coordinates, any curve along the sphere must satisfy $\sum_i \eta_i \dot{\eta}_i = 0$. By itself it has nothing to do with the fact that $\ell$ is geodesic. The "slick" derivation that I referred to is essentially what I wrote here, but cast in the language of information geometry where for some reason one prefers the coordinates $\xi$ instead of $\eta$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks @Willie Wong. You have used the particular parameterization $\pi$ to derive the desired equation. Don't we have to prove that the desired equation can be derived independent of the parameterization? $\endgroup$ – Ashok May 20 '14 at 14:11
  • $\begingroup$ Somehow Amari says that $\dot l_t$ can be considered as a tangent vector of the curve at $\gamma_t$. Using this fact one may get the desired equation $\ddot l_t+i_t=0$ rather easily? $\endgroup$ – Ashok May 21 '14 at 1:27
  • $\begingroup$ @Ashok: about your first comment: I have no idea what you are asking above. The functions $\pi_k$ are just your function $p_\xi(x)$. What do you mean by parametrisation? These functions are explicitly defined and they are used to define the metric $g$. I don't see any free parametrisation floating around. $\endgroup$ – Willie Wong May 21 '14 at 8:30
  • $\begingroup$ Let me edit the answer to address your second comment. $\endgroup$ – Willie Wong May 21 '14 at 8:43
  • $\begingroup$ What I meant was the coordinate system: The coordinate system you have used here is the one $p\mapsto \xi$ given by $p(x_i)=\xi_i$ for $i=1,\dots,n$ and $p(x_i)=1-\sum_j \xi_j$ for $i=0$. Your derivation depend on this particular coordinate system. For example, you have used the fact that $\partial_k \pi_m = \delta_{km}$ for $m>0$ and $-1$ otherwise. Would the desired equation change if we use a different coordinate system, say $p\mapsto \theta$ where $p(x)$ is a different function of $\theta$? $\endgroup$ – Ashok May 21 '14 at 9:11
0
$\begingroup$

I do not see why your metric is a sensible object, because it depends highly on your choice of coordinates (I am not even sure that your formula defines a metric, but I may be overlooking something). Furthermore, it is not clear at all why your Christoffel symbols are well-defined, as they have to transform in a certain way under coordinate change to actually give a connection.

However, if we assume that they actually do define a connection, you just have to check the definition: A path $\gamma$ is a geodesic if it fulfills the geodesic equation $$\frac{\nabla}{d t} \dot{\gamma}(t) \equiv 0,$$ where in terms of Christoffel symbols, $$\frac{\nabla}{d t} \dot{\gamma}(t) = \ddot{\gamma}(t) + \Gamma_{ij}^k \,\dot{\gamma}(t)\,\dot{\gamma}(t).$$ If the formula from the paper is true in your situation, you can just verify it by explicitly computing every term according to your definitions.

However, there should be an invariant (coordinate-independent) definition of the connection, which should give way more insight in what this actually means (and shows that everything is indeed well-defined).


Earlier Post:

Your notation is quite strange to me, and very non-standard, at least coming from differential geometry.

You go though some length to define the interior of the $n$-simplex $$\mathrm{int}(\Delta^n) := \bigl\{ x \in \mathbb{R}^{n+1} \mid \sum_{i=0}^n \xi = 1 ~~\text{and} ~~ x_i > 0 ~~\text{for all}~~ i\bigr\}.$$ This is an $n$-dimensional submanifold of $\mathbb{R}^{n+1}$ and it is flat with respect to the induced metric.

However, apparently you want to define a non-standard metric on this manifold. This is where things get incomprehensible for me. First of all, what is $\xi$? You write it is "the" global coordinate system. Obviously there are various coordinate symstems on $\mathrm{int}(\Delta^n)$ (e.g. those given by one of the projection maps along one of the axes), but I do not see how there would be a canonical one.

Secondly, what is $p_\xi$? You write that $\mathcal{P} = \{p_\xi\}$, but I do not see how to interpret this as a meaningful set. Then again, $p_\xi$ seems to be a scalar, because you take its $\log$ later on.

Thirdly, the formula for Christoffel symbols generally looks different from the one you write. Probably it is right in this context, but as I said, I have no idea what $p_\xi$ should be.

And even then, you define $l_t = \log \dot{\gamma}(t)$, which does not make sense, as $\gamma$ is a path in a manifold, where you cannot take the log of. Even if you identify $\dot{\gamma}(t)$ with the corresponding coordinates in you coordinate system, that is still a vector in $\mathbb{R}^n$, where I do not know how to take the $\log$ of.

So, quite frankly, the reason that you didn't get a response on math.StackExchange might be that you do not define a single thing you write.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Those who know bit of information geometry know the context. The metric and the connection I have defined are most standard there. Any way, I've elaborated a little more. Please have a look at the edited question. $\endgroup$ – Ashok May 20 '14 at 1:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.