Asymptotic bound on the total variation distance between a standard multivariate normal and a simple mixture

Question

Let $P = N(\vec{0}, I^d)$ be a standard multivariate Gaussian distribution in $d$ dimensions. Let $Q$ be distributed the same as $P$, except that samples from $Q$ have one of their coordinates, chosen uniformly at random from $1$ to $d$, distributed as $N(\mu, \sigma^2)$ instead.

That is, $Q$ is a mixture of $d$ multivariate Gaussians, each with weight $\frac1d$. The $i$th mixture component has mean $\vec{0} + \vec{e}_i \cdot \mu$ and variance with diagonal $\vec{1} + \vec{e}_i \cdot (\sigma^2 - 1)$.

I'd like to bound the total variation distance between $P$ and $Q$, as a function of $d$ (when $\mu$ and $\sigma$ are constants that do not depend on $d$).

My intuition is that $Q$ is very close to a multivariate Gaussian $N(\frac{\vec{\mu}}{d}, I^d \cdot (1+\frac{\sigma^2}{d}+ o(\frac1d))$, which would give a Kullback–Leibler divergence of $D_{KL}(P || Q) = O(\frac1d)$ and thus a total variation distance of $O(\frac{1}{\sqrt d})$.

The case where $\sigma^2=1$ might be easier and sufficient for what I need. Any suggestions on how I could bound the TV distance between P and Q?

Answer 1

$\newcommand{\m}{\vec{\mu}} \newcommand{\e}{\vec{e}} \newcommand{\x}{\vec{x}}$ Let $n:=d$. Assume that $\sigma=1$ and $\mu\ne0$. Then for the densities \begin{equation*} f_Q(\x)=(2\pi)^{-n/2}\,\frac1n\,\sum_1^n e^{-|\x-\mu\e_i|^2/2} \end{equation*} and \begin{equation*} f_P(\x)=(2\pi)^{-n/2}\,e^{-|\x|^2/2} \end{equation*} of $P$ and $Q$ we have \begin{equation*} f_Q(\x)<f_P(\x)\iff \sum_1^n e^{\mu x_i}<ne^{\mu^2/2}, \end{equation*} where $\x=(x_1,\dots,x_n)\in\mathbb R^n$ and $|\cdot|$ denotes the Euclidean norm. Hence, \begin{equation*} d_{TV}(P,Q)=p_1-p_2, \tag{1} \end{equation*} where \begin{equation*} p_1:=P(S_n<ne^{\mu^2/2}),\quad p_2:=P(T_n<ne^{\mu^2/2}), \end{equation*} \begin{equation*} S_n:=\sum_1^n U_i,\quad T_n:=S_{n-1}+V_n, \end{equation*} \begin{equation*} U_i:=e^{\mu Z_i},\quad V_n:=e^{\mu(Z_n+\mu)}=e^{\mu^2}U_n, \end{equation*} and the $Z_i$'s are iid standard normal random variables. Note that \begin{equation*} ES_n=na,\quad ET_n=na+O(1),\quad Var\, S_n=nb^2,\quad Var\,T_n=nb^2+O(1), \end{equation*} where $a:=e^{\mu^2/2}$ and $b^2:=e^{2\mu^2}-e^{\mu^2}$. Also, $E|U_1|^3<\infty$. So, letting $\Phi$ denote the standard normal cumulative distribution function and using the Berry--Esseen inequality, we have \begin{equation*} p_1=P(S_n<na)=\Phi(0)+O(1/\sqrt n)=\frac12+O(1/\sqrt n) \end{equation*} and \begin{multline*} p_2=P(T_n<na)=\Phi\Big(\frac{na-ET_n}{\sqrt{Var\,T_n}}\Big)+O(1/\sqrt n) \\ =\Phi\Big(\frac{O(1)}{\sqrt{nb^2+O(1)}}\Big)+O(1/\sqrt n) =\frac12+O(1/\sqrt n) \end{multline*} Thus, by (1), we have \begin{equation*} d_{TV}(P,Q)=O(1/\sqrt n), \end{equation*} as conjectured.