Does multiplication increase entropy?
The Shannon entropy of a number $k$ in binary digits is defined as $$ H = -\log(\frac{a}{l})\cdot\frac{a}{l} - \log(1-\frac{a}{l})\cdot (1-\frac{a}{l})$$ where $l = 1+\text{ floor }(\frac{\log(k)}{\log(2)})$ is the number of binary digits of $k$ and $a$ is the number of $1$-s in the binary expansion of $k$. So we view the number $k$ as a "random variable".
Suppose that $n,m$ are uniformly randomly chosen in the interval $1 \le N$.
Hypothesis 1):
$H_{m \cdot n}$ is "significantly" larger then $H_n$.
Hypothesis 2):
$H_{m + n}$ is not "significantly" larger then $H_n$.
Here is some empirical statistical test indicating that multiplication increases entropy, but addition does not:
from collections import Counter
def entropyOfCounter(c):
S = 0
for k in c.keys():
S += c[k]
prob = []
for k in c.keys():
prob.append(c[k]/S)
H = -sum([ p*log(p,2) for p in prob]).N()
return H
def HH(l):
return entropyOfCounter(Counter(l))
N = 10^4
HN = []
HmXn = []
HmPn = []
for k in range(N):
n = randint(1,17^50)
m = randint(1,17^50)
Hn = HH(Integer(n).digits(2))
Hm = HH(Integer(m).digits(2))
HmXn.append(HH(Integer(n*m).digits(2)))
HmPn.append(HH(Integer(n+m).digits(2)))
HN.append(Hn)
X = mean(HN)
Y = mean(HmPn)
Z = mean(HmXn)
n = len(HN)
m = n
SX2 = variance(HN)
SY2 = variance(HmPn)
SZ2 = variance(HmXn)
SXY2 = ((n-1)*SX2 + (m-1)*SY2)/(n+m-2)
SXZ2 = ((n-1)*SX2 + (m-1)*SZ2)/(n+m-2)
TXY = sqrt((m*n)/(n+m)).N()*(X-Y)/sqrt(SXY2).N()
TXZ = sqrt((m*n)/(n+m)).N()*(X-Z)/sqrt(SXZ2).N()
print TXY,TXZ,n+m-2
Output: -1.43265218355297 -32.5323306851490 19998
The second case (multiplication) increases entropy significantly. The first case ( addition) does not.
Is there a way to give a heuristic explanation why this is so in general (if it is), or is this empirical obervation in general $1 \le N$ wrong?
Related:
https://math.stackexchange.com/questions/3275096/does-entropy-increase-when-multiplying-two-numbers