Parameterization of exponential family

Question

Let $\{\mathbb{P}_{\theta}\}_{\theta}$ be an exponential family of probability measures, all with finite mean. Under what conditions is the parameterization map $\theta\mapsto \mathbb{P}_{\theta}$ continuous with respect to the Wasserstein-$1$ distance?

Answer 1

$\newcommand\om\omega\newcommand\Om\Omega\newcommand\ep\epsilon\newcommand\R{\mathbb R}$Let $(P_t)_{t\in T}$ be an exponential family over a separable complete metric space $(X,d)$, where $T$ is an open subset of $\mathbb R^k$ and $t=(t_1,\dots,t_k)$ is a natural parameter. So, for all $t\in T$, the density $p_t$ of $P_t$ with respect to Borel measure $\mu$ on $X$ is given by the formula \begin{equation*} p_t(x)=c(t)\exp\{t\cdot U(x)\}\tag{1} \end{equation*} for all $x\in X$, where $U\colon X\to\mathbb R^k$ is a Borel-measurable function, $\cdot$ denotes the dot product on $\mathbb R^k$, \begin{equation*} c(t):=1/I(t),\quad I(t):=\int_X\mu(dx)\exp\{t\cdot U(x)\}\in(0,\infty). \tag{2} \end{equation*}

Assume that, for some (and hence for all) $x_0\in X$, the "first moment" \begin{equation} m_t(x_0):=\int_X P_t(dx)d(x_0,x)=\int_X \mu(dx)p_t(x)d(x_0,x)<\infty\tag{2a} \end{equation} for all $t\in T$.

Then the parameterization map $U\ni t\mapsto P_t$ is continuous with respect to the Wasserstein-1 distance $W_1$.

Indeed, by the penultimate paragraph of the introduction section, if suffices to show that for each $t\in T$ \begin{equation*} \int_X \mu(dx)p_s(x)f(x)\to \int_X \mu(dx)p_t(x)f(x)\tag{3} \end{equation*} as $s\to t$, where $f\colon X\to\mathbb R$ such that \begin{equation*} C:=\sup_{x\in X}|f(x)|/(1+d(x_0,x))<\infty \tag{3a} \end{equation*} (for some (and hence for all) $x_0\in X$).

To verify (3), take any $t=(t_1,\dots,t_k)$ in the open set $T$. Take any real $\ep>0$ such that $t+\ep\om\in T$ for all $\om=(\om_1,\dots,\om_k)\in\Om:=\{-1,1\}^k$. Take then any $s\in T$ such that $\max_{j=1}^k|s_j- t_j|\le\ep$ and any $u\in\R^k$. Then \begin{align*}\exp\{s\cdot u\}/\exp\{t\cdot u\}&=\exp\{(s-t)\cdot u\} \\ &\le\max_{\om\in\Om}\exp\{\ep\om\cdot u\} \\ &\le\sum_{\om\in\Om}\exp\{\ep\om\cdot u\}; \end{align*} the first inequality in the above display follows because the function $\exp$ is convex and the condition $\max_{j=1}^k|s_j- t_j|\le\ep$ implies that the point $s-t$ is in the convex hull of the set $\{\ep\om\colon\om\in\Om\}$. So, for all $x\in X$ \begin{equation*} \exp\{s\cdot U(x)\}\le g(x):=\sum_{\om\in\Om}\exp\{( t+\ep\om)\cdot U(x)\}. \tag{5} \end{equation*} Moreover, by (2), \begin{equation*} \int_X\mu(dx)g(x)=\sum_{\om\in\Om}I(t+\ep\om)<\infty. \end{equation*} So, by the dominated convergence theorem, $I(s)\to I(t)$ and hence $c(s)\to c(t)$ as $s\to t$. So, choosing $\ep$ small enough, we have \begin{equation*} c(s)\le 2c(t). \tag{6} \end{equation*}

In view of (1) and (6), inequality (5) yields \begin{equation*} p_s(x)\le c(s)\sum_{\om\in\Om}\frac{p_{t+\ep\om}(x)}{c(t+\ep\om)} \le K\sum_{\om\in\Om}p_{t+\ep\om}(x), \end{equation*} where \begin{equation*} K:=2c(t)\max_{\om\in\Om}\frac1{c(t+\ep\om)}<\infty. \end{equation*} So, by (3a), \begin{equation*} p_s(x)|f(x)|\le h(x):=KC\sum_{\om\in\Om}p_{t+\ep\om}(x)(1+d(x_0,x)). \end{equation*} Moreover, by (2a), \begin{equation*} \int_X \mu(dx)h(x)=KC\sum_{\om\in\Om}(1+m_{t+\ep\om}(x_0))<\infty. \end{equation*} So, again by the the dominated convergence theorem, (3) follows. $\Box$