Edit: I simplified the example to a canonical case for clarity.
Given a integral $\int_{\Omega}{g(x)}$ with a well-posed integrand $g(x)$ defined on some space $\Omega$, one can integrate it successfully with a Markov Chain Monte Carlo method, in particular using Metropolis-Hastings method. Detailed balance and ergodicity are achieved with any well-posed (samplable) integrand. The value of the desired integral is known to always exist and be finite, even in presence of delta distributions in the integrand. This is given.
In the case of interest, the integrand $f(x)=\delta_{x_0}(x)+g(x)$ consists of a delta distribution at some unknown location $x_0\in \Omega$ plus some regular (well-posed) non-zero integrand $g(x)$. This delta distribution cannot be sampled explicitly or found by numerical optimization (given). This makes such an integrand unsamplable with random walk or probabilistic sampling.
I mollify (approximate to the identity) this delta distribution using some mollifier (normalized smooth function $\phi_\epsilon(x)=\epsilon^{-1}\phi\left(\frac{x}{\epsilon}\right)$ with some bandwidth $\epsilon$). This leads to a tempered integrand $f_\epsilon(x)=\phi_\epsilon(x)+g(x)$. During the integration, at every step $n$, I gradually shrink the parameter $\epsilon_n$ to zero in order to achieve $f_{\epsilon_n} \to f=\delta_{x_0}(x)+g(x)$ as $n \to \infty$ in spirit of serial tempering. Thus I expect the integration method to be consistent, i.e. to give the proper answer in the limit.
Thus I have two rather similar questions:
- Would the integral converge to proper value (which is $\int_{\Omega}{g(x)}+1$ in this case) with this method if the integrand has a delta distribution? What confuses me is that the chain either tends to stuck in the vicinity of the delta distribution or would otherwise just miss it, as the mode of it becomes narrower when $n$ is large.
- What should be the cool-down schedule of the integrand $f_{\epsilon_n}$, i.e. the shrinkage rate of the sequence $\{ \epsilon_n \}$, in order to guarantee the consistent convergence of the MCMC estimate? In other words, what are the conditions for the cool-down schedule that allows the integral to converge before the integrand $f_{\epsilon_n}$ becomes unsamplable?