If two different probability distributions have identical moments, are they equal? I suspect not, but I would guess they are "mostly" equal, for example, on everything but a set of measure zero. Does anyone know an example of two different probability distributions with identical moments? The less pathological the better. Edit: Is it unconditionally true if I specialize to discrete distributions?
And a related question: Suppose I ask the same question about Renyi entropies. Recall that the Renyi entropy is defined for all a
≥ 0$a \geq 0$ by
Ha(p) = log(∑j pja)/(1-a)$$ H_a(p) = \frac{\log \left( \sum_j p_j^a \right)}{1-a} $$
You can define a
=0,1,∞$a = 0, 1, \infty$ by taking suitable limits of this formula. Are two distributions with identical Renyi entropies (for all values of the parameter a
$a$) actually equal? How "rigid" is this result? If I allow two Renyi entropies of distributions p
$p$ and q
$q$ to differ by at most some small ε $\epsilon$ independent of a
$a$, then can I put an upper bound on, say, || p - q ||1$||p-q||_1$ in terms of ε$\epsilon$? What can be said in the case of discrete distributions?