What can we say about the order of convergence of a critical point of Gaussian mixture density to its limit when the parameter $h$ goes to $0?$

Question

Density of Gaussian mixture with $n$ components is given by:

$$f(x):=C \sum_{i=1}^{n}e^{-\frac{1}{2}||\frac{x-x_i}{h}||^2}, x_i \in \mathbb{R}^d, h > 0$$

where $C$ is a normalization constant ensuring the total integral over $\mathbb{R}^d$ equals one.

Its critical points are implicitly given by:

$$x^{*}_h= \frac{ \sum_{i=1}^{n}e^{-\frac{1}{2}||\frac{x^{*}_h-x_i}{h}||^2}x_i }{ \sum_{i=1}^{n}e^{-\frac{1}{2}||\frac{ x^{*}_h -x_i}{h}||^2} }$$

Assuming that for all small positive $h, ||x^{*}_h - x_1|| < ||x^{*}_h - x_i|| \forall i \ge 2,$ it's easy to see that:

$$x^{*}_h \to x_1, h \to 0.$$

My question is about the order of this convergence in terms of $h,$ i.e. what can we say about the order in terms of $h$ that the quantity $||x^{*}_h - x_1||$ converges to $0$ as $h\to 0?$ Is it for example $O(h), o(h), \Theta(h)$ etc? A precise bound will be great, but some others would be appreciated as well!

Answer 1

In general, it is not true at all that $x^*_h\to x_1$ as $h\downarrow0$. Indeed, suppose that $n=3$, $x_1=2$, $x_2=0$, $x_3=-1$.

Then for small $h>0$ there will be a critical point $x^*_h$ close to the midpoint $1$ between $x_1=2$ and $x_2=0$, with $x_1=2$ being the closest to $x^*_h$ among the points $x_1,x_2,x_3$ -- because $f'(1)=-2Ce^{-2/h^2}/h^2<0$. However, $x^*_h-x_1$ will be close to $1-2=-1$ and not to $0$.

---

For an illustration, here are the graphs $\{(x,f(x))\colon x\in[-1.5,2.5]\}$ and $\{(x,f(x))\colon x\in[1-\frac12\,10^{-7},1+\frac12\,10^{-7}]\}$ for $h=3/10$: