Are two probability distributions uniquely constrained by the sum of their p-norms? - MathOverflow most recent 30 from http://mathoverflow.net 2013-06-19T04:35:52Z http://mathoverflow.net/feeds/question/8388 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/8388/are-two-probability-distributions-uniquely-constrained-by-the-sum-of-their-p-norm Are two probability distributions uniquely constrained by the sum of their p-norms? Steve Flammia 2009-12-09T19:49:41Z 2012-08-22T12:09:06Z <p>Let A, B and C be finitely supported probability distributions with at most d nonzero probabilities each. Now consider the following simultaneous equations using p-norms, for each value of p&ge;1, given by</p> <p>||A||<sub>p</sub> + ||B||<sub>p</sub> = ||C||<sub>p</sub></p> <p>where A, B and C are still non-negative, but we relax normalization on A and B. Imagine that C is fixed and, without loss of generality, normalized. We want to solve for A and B.</p> <p>First, note that one obvious family of solutions is </p> <p>A = (1-x) C , B = x C , 0&le;x&le;1 .</p> <p>Question: Ignoring the obvious permutation symmetries, are these the <em>only</em> solutions?</p> <p>Edit: By p-norm, I mean the vector p-norm: ||A||<sub>p</sub> = (&sum;<sub>j</sub> |a<sub>j</sub>|<sup>p</sup> )<sup>1/p</sup>. Although we don't really need the absolute values, since the a<sub>j</sub> are all non-negative.</p> http://mathoverflow.net/questions/8388/are-two-probability-distributions-uniquely-constrained-by-the-sum-of-their-p-norm/8688#8688 Answer by Alekk for Are two probability distributions uniquely constrained by the sum of their p-norms? Alekk 2009-12-12T16:33:11Z 2009-12-15T14:40:25Z <p>if you suppose that all the $a_i$, all the $b_i$, and all the $c_i$ are distinct, can't you do that by induction ? One can assume wlog that $a_1 > \ldots > a_d > 0$ and $b_1 > \ldots > b_d > 0$ etc.. so that taking $p \to \infty$ you can see that $a_1+b_1=c_1$. Hence $$a_1\left(1+\sum_2^d \left(\frac{a_k}{a_1}\right)^p\right)^{1/p} + b_1\left(1+\sum_2^d \left(\frac{b_k}{b_1}\right)^p\right)^{1/p} = c_1\left(1+\sum_2^d \left(\frac{c_k}{c_1}\right)^p\right)^{1/p}$$ and a Taylor expansion for $p \to \infty$ tells you that $\frac{a_2}{a_1}=\frac{b_2}{b_1}=\frac{c_2}{c_1}$. Continuing this way, one can see that the only family of solutions is $A=\lambda C$ and $B=(1-\lambda) C$. Too simple to be true ?</p> http://mathoverflow.net/questions/8388/are-two-probability-distributions-uniquely-constrained-by-the-sum-of-their-p-norm/9408#9408 Answer by Greg Kuperberg for Are two probability distributions uniquely constrained by the sum of their p-norms? Greg Kuperberg 2009-12-20T06:07:45Z 2012-08-22T12:09:06Z <p>Here is a proof that Steve's rescaling gives you all solutions, together with the trivial operation of permuting the components of $A$, $B$, and $C$ if you view them as vectors with positive coeifficients. (If you view them this way, then Steve's notation $||A||_p$ is just the usual $p$-norm.)</p> <p>I first tried what Alekk tried: You can take the limit as $p \to \infty$ and eventually obtain certain power series expansions in $1/p$. Or you can take the limit $p \to 0$ and obtain certain power series expansions in $p$. The problem with both approaches is that the information in the terms of these expansions is complicated. To help understand the second limit, I observed that the two sides of Steve's equation are analytic in $p$, but it only helped so much.</p> <p>Then I realized that when you have a complex analytic function of one variable, you can get a lot of information from looking at singularities. So let's look at that. Let $\alpha_k = \ln a_k$, so that <code>$$||A||_p = \exp\left( \frac{\ln \bigl[\exp(\alpha_1 p) + \exp(\alpha_2 p) + \cdots + \exp(\alpha_d p) \bigr]}{p} \right).$$</code> The expression inside the logarithm has been called an exponential polynomial in the literature, which I'll call $a(p)$. As indicated, $||A||_p$ has a logarithmic singularity when $a(p) = 0$. $||A||_p$ has another kind of singularity when $p = 0$, but won't matter for anything. Also $a(p)$ is an entire function, which means in particular that it is univalent and has isolated zeroes. Also, none of the zeroes of $a(p)$ are on the real axis. Let $b(p)$ and $c(p)$ be the corresponding exponential polynomials for $B$ and $C$.</p> <p>Suppose that you follow a loop that starts on the positive real axis, encircles an $m$-fold zero of $a(p)$ at $p_0$, and then retraces to its starting point. Then the value of <code>$||A||_p$</code>, which is non-zero for $p > 0$, gains a factor of $\exp(2m\pi i/p_0)$. Thus Steve's equation is not consistent unless all three of $a(p)$, $b(p)$ and, $c(p)$ have the same zeroes with the same multiplicity. (Since $\exp(2m\pi i/p_0)$ cannot have norm 1, geometric sequences with this ratio but with different values of $m$ are linearly independent.)</p> <p>At this point, the problem is solved by a very interesting paper of Ritt, <a href="http://www.jstor.org/pss/1989556" rel="nofollow">On the zeros of exponential polynomials</a>. Ritt reviews certain results of Tamarkin, Polya, and Schwengler, which imply in particular that if an exponential polynomial $f(z)$ does not have any zeroes, then it is a monomial $f_\alpha \exp(\alpha z)$. Ritt's own theorem is that if $f(z)$ and $g(z)$ are exponential polynomials, and if the roots of $f(z)$ are all roots of $g(z)$ (with multiplicity), then their ratio is another exponential polynomial. Thus in our situation $a(p)$, $b(p)$, and $c(p)$ are all proportional up to a constant and an exponential factor. Thus, $A$, $B$, and $C$ must be the same vectors up to permutation, repetition, and rescaling of the coordinates. Repetition is an operation that hasn't yet been analyzed. If $A^{\oplus n}$ denotes the $n$-fold repetition of $A$, then <code>$||A^{\oplus n}||_p = n^{1/p}||A||_p$</code>. Again, since geometric sequences with distinct ratios are linearly independent, Steve's equation is not consistent if $A$, $B$, and $C$ are repetitions of the same vector by different amounts.</p> <p>The same argument works for the generalized equation <code>$$x_1||A_1||_p + x_2||A_2||_p + \cdots + x_n||A_n||_p = 0.$$</code> The result is that any such linear dependence trivializes, after rescaling the vectors and permuting their coordinates.</p> <p><b>Update</b> (by <em>J.O'Rourke</em>): Greg's paper based on this solution was just published: </p> <blockquote> <p>"Norms as a function of $p$ are linearly independent in finite dimensions," <em>Amer. Math. Monthly</em>, Vol. 119, No. 7, Aug-Sep 2012, pp. 601-3 (<a href="http://www.jstor.org/stable/10.4169/amer.math.monthly.119.07.601" rel="nofollow">JSTOR link</a>).</p> </blockquote>