Mixture of gaussian density agree with another gaussian on positive measure

Question

I noticed this post. But still I'd like to follow up with a specific case I have in mind. Say $p(x| \theta)$ is the density of a Gaussian distribution on $\mathbb{R}^n$ with mean $\theta$ and known covariance $\Sigma$. Let $$f(x) = \int_{\Theta} p(x| \theta) \Lambda(d\theta),$$ where $\Theta$ is a proper subset of $\mathbb{R}^n$, and $\Lambda$ is an arbitrary probability measure over $(\Theta, \mathcal{B}(\Theta))$, not necessarily continuous w.r.t. the Lebesgue measure. Let $g(x)$ be the density of another Gaussian on $\mathbb{R}^n$ with mean $\mu$ and known covariance $\Sigma + \Psi$. Is it possible for $g(x)$ and $f(x)$ to agree on a set of positive Lebesgue measure?

Answer 1

$\newcommand\R{\mathbb R}\renewcommand\th{\theta}\newcommand{\Th}{\Theta}\newcommand{\Si}{\Sigma} \newcommand\La\Lambda\newcommand{\C}{\mathbb C}$The answer is: This will be so if (and only if) $\Lambda$ itself is a (possibly degenerate) Gaussian distribution.

Let us prove the "only if" part. Here it does not really matter whether it is a priori assumed that $\Th$ is a proper subset of $\R^n$. Also, by substitutions $x\leftrightarrow\Si^{1/2}x$ and $\th\leftrightarrow\Si^{1/2}\th$, without loss of generality $\Si=I_n$, the identity matrix. So,
\begin{equation*} f(x)=(2\pi)^{-n/2}\int_{\R^n}\exp\Big(-\frac12\sum_{j=1}^n(x_j-\th_j)^2\Big)\,\La(d\th) \end{equation*} for $x=(x_1,\dots,x_n)\in\R^n$. Since \begin{equation*} \sum_{j=1}^n(x_j+iy_j-\th_j)^2=\sum_{j=1}^n(x_j-\th_j)^2-\sum_{j=1}^n y_j^2 +2i\sum_{j=1}^n(x_j-\th_j)y_j, \tag{2}\label{2} \end{equation*} the function $f$ can be extended to a holomorphic function on $\C^n$. Similarly, the function $g$ can be extended to a holomorphic function on $\C^n$.

So, $f$ and $g$ are real-analytic on each (straight) line in $\R^n$.

Suppose now $f=g$ on some subset $A$ of $\R^n$ of Lebesgue measure $|A|>0$. Without loss of generality, the set $A$ is bounded.

Take any $x\in\R^n$. Then \begin{equation*} 0<|A|=|A-x|=\int_{S^{n-1}}du\,\int_0^\infty r^{n-1}\,dr\,1(ru\in A-x), \end{equation*} where $\int_{S^{n-1}}du$ is the integral with respect to the surface measure on the unit sphere $S^{n-1}$ in $\R^n$. So, by the Tonelli theorem, there is some $u_x\in S^{n-1}$ such that $\int_0^\infty r^{n-1}\,dr\,1(ru_x\in A-x)>0$ and hence \begin{equation*} |A\cap(x+\R_+ u_x)|=\int_0^\infty dr\,1(x+ru_x\in A)=\int_0^\infty dr\,1(ru_x\in A-x)>0. \end{equation*} So, the set $A\cap(x+\R_+ u_x)$ is an infinite and bounded subset of the line $x+\R u_x$, and $f=g$ on this infinite bounded subset of a line. Recalling that $f$ and $g$ are real-analytic on each line in $\R^n$, we conclude that $f=g$ on the line $x+\R u_x$, and hence $f(x)=g(x)$.

So, $f=g$ (on the entire space $\R^n$), which can be rewritten as the identity \begin{equation*} e^{-\|x\|^2/2}\int_{\R^n}\exp\Big(\sum_{j=1}^n \th_j x_j\Big)\,L(d\th) =C\exp\big(-x^\top Bx+x^\top\mu\big) \end{equation*} for all $x\in\R^n$, where $\|\cdot\|$ is the Euclidean norm, $$L(d\th):=e^{-\|\th\|^2/2}\La(d\th),$$ $B$ is some positive-definite $n\times n$ real matrix, $\mu$ is some vector in $\R^n$, and $C:=L(\R^n)=(2\pi)^{n/2}f(0)$. So, for (joint) moment generating function (mgf) $M_{L/C}$ of the probability measure $L/C$, some symmetric matrix $R$, and all $x\in\R^n$ we have \begin{equation*} M_{L/C}(x)=\exp\big(-x^\top Rx+x^\top\mu\big) \end{equation*} Any mgf is log convex. So, $R$ must be positive semidefinite. Thus, $M_{L/C}$ is the mgf of a (possibly degenerate) Gaussian distribution. So, $L/C$ is a (possibly degenerate) Gaussian distribution. So, $\La$ is a (possibly degenerate) Gaussian distribution.

Vice versa, if $\La$ is a (possibly degenerate) Gaussian distribution, then $f$ is clearly a Gaussian distribution. $\quad\Box$