Are two probability distributions uniquely constrained by the sum of their p-norms?

2009-12-09T19:49:41Z

Let A, B and C be finitely supported probability distributions with at most d nonzero probabilities each. Now consider the following simultaneous equations using p-norms, for each value of p≥1, given by

||A||_p + ||B||_p = ||C||_p

where A, B and C are still non-negative, but we relax normalization on A and B. Imagine that C is fixed and, without loss of generality, normalized. We want to solve for A and B.

First, note that one obvious family of solutions is

A = (1-x) C , B = x C , 0≤x≤1 .

Question: Ignoring the obvious permutation symmetries, are these the only solutions?

Edit: By p-norm, I mean the vector p-norm: ||A||_p = (∑_j |a_j|^p )^1/p. Although we don't really need the absolute values, since the a_j are all non-negative.

Answer by Alekk for Are two probability distributions uniquely constrained by the sum of their p-norms?

2009-12-12T16:33:11Z

if you suppose that all the $a_i$, all the $b_i$, and all the $c_i$ are distinct, can't you do that by induction ? One can assume wlog that $a_1 > \ldots > a_d > 0$ and $b_1 > \ldots > b_d > 0$ etc.. so that taking $p \to \infty$ you can see that $a_1+b_1=c_1$. Hence $$a_1\left(1+\sum_2^d \left(\frac{a_k}{a_1}\right)^p\right)^{1/p} + b_1\left(1+\sum_2^d \left(\frac{b_k}{b_1}\right)^p\right)^{1/p} = c_1\left(1+\sum_2^d \left(\frac{c_k}{c_1}\right)^p\right)^{1/p}$$ and a Taylor expansion for $p \to \infty$ tells you that $\frac{a_2}{a_1}=\frac{b_2}{b_1}=\frac{c_2}{c_1}$. Continuing this way, one can see that the only family of solutions is $A=\lambda C$ and $B=(1-\lambda) C$. Too simple to be true ?

Answer by Greg Kuperberg for Are two probability distributions uniquely constrained by the sum of their p-norms?

2009-12-20T06:07:45Z

Here is a proof that Steve's rescaling gives you all solutions, together with the trivial operation of permuting the components of $A$, $B$, and $C$ if you view them as vectors with positive coeifficients. (If you view them this way, then Steve's notation $||A||_p$ is just the usual $p$-norm.)

I first tried what Alekk tried: You can take the limit as $p \to \infty$ and eventually obtain certain power series expansions in $1/p$. Or you can take the limit $p \to 0$ and obtain certain power series expansions in $p$. The problem with both approaches is that the information in the terms of these expansions is complicated. To help understand the second limit, I observed that the two sides of Steve's equation are analytic in $p$, but it only helped so much.

Then I realized that when you have a complex analytic function of one variable, you can get a lot of information from looking at singularities. So let's look at that. Let $\alpha_k = \ln a_k$, so that $$||A||_p = \exp\left( \frac{\ln \bigl[\exp(\alpha_1 p) + \exp(\alpha_2 p) + \cdots + \exp(\alpha_d p) \bigr]}{p} \right).$$ The expression inside the logarithm has been called an exponential polynomial in the literature, which I'll call $a(p)$. As indicated, $||A||_p$ has a logarithmic singularity when $a(p) = 0$. $||A||_p$ has another kind of singularity when $p = 0$, but won't matter for anything. Also $a(p)$ is an entire function, which means in particular that it is univalent and has isolated zeroes. Also, none of the zeroes of $a(p)$ are on the real axis. Let $b(p)$ and $c(p)$ be the corresponding exponential polynomials for $B$ and $C$.

Suppose that you follow a loop that starts on the positive real axis, encircles an $m$-fold zero of $a(p)$ at $p_0$, and then retraces to its starting point. Then the value of $||A||_p$ , which is non-zero for $p > 0$, gains a factor of $\exp(2m\pi i/p_0)$. Thus Steve's equation is not consistent unless all three of $a(p)$, $b(p)$ and, $c(p)$ have the same zeroes with the same multiplicity. (Since $\exp(2m\pi i/p_0)$ cannot have norm 1, geometric sequences with this ratio but with different values of $m$ are linearly independent.)

At this point, the problem is solved by a very interesting paper of Ritt, On the zeros of exponential polynomials. Ritt reviews certain results of Tamarkin, Polya, and Schwengler, which imply in particular that if an exponential polynomial $f(z)$ does not have any zeroes, then it is a monomial $f_\alpha \exp(\alpha z)$. Ritt's own theorem is that if $f(z)$ and $g(z)$ are exponential polynomials, and if the roots of $f(z)$ are all roots of $g(z)$ (with multiplicity), then their ratio is another exponential polynomial. Thus in our situation $a(p)$, $b(p)$, and $c(p)$ are all proportional up to a constant and an exponential factor. Thus, $A$, $B$, and $C$ must be the same vectors up to permutation, repetition, and rescaling of the coordinates. Repetition is an operation that hasn't yet been analyzed. If $A^{\oplus n}$ denotes the $n$-fold repetition of $A$, then $||A^{\oplus n}||_p = n^{1/p}||A||_p$ . Again, since geometric sequences with distinct ratios are linearly independent, Steve's equation is not consistent if $A$, $B$, and $C$ are repetitions of the same vector by different amounts.

The same argument works for the generalized equation $$x_1||A_1||_p + x_2||A_2||_p + \cdots + x_n||A_n||_p = 0.$$ The result is that any such linear dependence trivializes, after rescaling the vectors and permuting their coordinates.

Update (by J.O'Rourke): Greg's paper based on this solution was just published:

"Norms as a function of $p$ are linearly independent in finite dimensions," Amer. Math. Monthly, Vol. 119, No. 7, Aug-Sep 2012, pp. 601-3 (JSTOR link).

Are two probability distributions uniquely constrained by the sum of their p-norms? - MathOverflow

Are two probability distributions uniquely constrained by the sum of their p-norms?

Answer by Alekk for Are two probability distributions uniquely constrained by the sum of their p-norms?

Answer by Greg Kuperberg for Are two probability distributions uniquely constrained by the sum of their p-norms?