5
$\begingroup$

Let us consider a probability distribution $(g_n)_{n \in \mathbb{N}}$ which we want to approximate by a mixture of $(f_n(\lambda))_{n \in \mathbb{N}}$ where $\lambda \in \mathbb{R}$ is a parameter.

Are there known techniques that allow one to find the mixture minimizing the $L^1$ norm: \begin{equation} \min_{p} \sum_{n=0}^{\infty} \left|g_n - \int \rm{d} \lambda \; p(\lambda) f_n(\lambda) \right| \end{equation} where $p(\lambda)$ is a normalized probability distribution?

The motivation of this problem is linked to experimental physics: ideally one would like to generate an experimental process characterized by the probability distribution $g$ but this is really not practical. What is really easy, however, is to generate an experimental process with the distribution $f(\lambda)$ where $\lambda$ is a tunable parameter. Therefore, the goal is to approximate $g$ as closely as possible with such a mixture of $f(\lambda)$, where the distance between the two distribution is computed with the $L^1$ norm, that is, I want to minimize the variation distance between the two distributions.

In the specific problem I consider, $f(\lambda)$ is a Poisson distribution with parameter $\lambda \geq 0$, but I really am interested in a general method to approach this problem-

Any pointer to the relevant literature would be greatly appreciated. Thanks a lot!

Improve this question
$\endgroup$
7
  • $\begingroup$ Can you motivate a bit more your problem (why do you need that in two lines, is it some sort of least favorable prior for simultaneous testing)? it is probability over $\mathbb{R}$ ? The norm you use in your sum is the $L^1$ norm between distribution right ? note that $p$ should have integral = 1. $\endgroup$
    – robin girard
    Commented Sep 1, 2010 at 11:02
  • $\begingroup$ I edited my question according to your remarks. $\endgroup$
    – Anthony Leverrier
    Commented Sep 1, 2010 at 12:08
  • 1
    $\begingroup$ Maybe it's better to bound $L^1$ distance by $L^2$, then Fourier analytic techniques can be used. That is also the strategy used in length minimization via energy minimization, well known to differential geometers. $\endgroup$
    – John Jiang
    Commented Sep 1, 2010 at 20:58
  • $\begingroup$ Unfortunately, it is really the $L^1$ distance which is relevant in my problem so I cannot switch from the $L^1$ to the $L^2$ distance. Furthermore, as the distributions are defined over $\mathbb{N}$, I cannot see how a bound on the $L^2$ distance could give any information concerning the $L^1$ distance? $\endgroup$
    – Anthony Leverrier
    Commented Sep 2, 2010 at 16:05
  • $\begingroup$ The iteration algorithm called Expectation-maximization is often suitable for approximations with mixitures, though it might not actually converge to the minimum you asked for. $\endgroup$
    – Zsbán Ambrus
    Commented Sep 16, 2010 at 14:39

1 Answer 1

Reset to default
2
$\begingroup$

perhaps for starters you could take $p$ to be supported on a finite number of points. then the constraints on $p$ become simple linear inequalities and you have a [convex - perhaps even linear] programming problem.

there has been attention in the statistical literature to fitting models using least absolute deviations, rather than least squares. [in the simplest case, the minimizer of

$$\sum_{i=1}^n |x_i - a|$$

is the sample median - rather than the sample mean one gets for $a$ using least squares.]

you could see if references in the monograph by yadolah dodge [L$_1$ statistical procedures and related topics, ims lecture notes - monograph series vol 31 1997] give anything useful.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the reference! Yes, I indeed started by taking $p$ supported on a finite number of points, and it works pretty well in practice. Still, it's a bit frustrating as a continuous density would be much more natural. $\endgroup$
    – Anthony Leverrier
    Commented Sep 16, 2010 at 16:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .