2
$\begingroup$

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Let entropy of $\mathcal{A}=\{a_i\}_{i=1}^m$ be given by $$H(\mathcal{A},m)=-\sum_{i=0}^mp_i\log_2p_i.$$

$H(\mathcal{A},m)$ has interesting properties. Example: discussion in Limiting Entropy of deterministic sequences - 1 shows following holds if $k>0$ as $m\in\Bbb N$ increases:

$$a_{i+1}=a_i+\Theta(\log^ka_i)\implies H(\mathcal{A,m})\mbox{ is unbounded}$$ $$a_{i+1}=a_i+\Theta(a_i^{\frac{1}{k}})\implies H(\mathcal{A,m})\mbox{ is bounded}$$

Here symbol $\Theta$ from Landau notations in complexity theory http://en.wikipedia.org/wiki/Big_O_notation#Family_of_Bachmann.E2.80.93Landau_notations.

Is there a physical interpretation of application of Shannon entropy in this situation? Does it mean we should be able to compress $a_i$ in finite number of bits or impossible to do in finite number of bits in total? This is true if $a_i$s could be thought of 'occurring' with probability $p_i$. This is true if there is an interpretation of $a_i$s as random variables. We generate probabilities from fixed integers which seems to make traditional compression interpretation shaky.

$\endgroup$
4
  • $\begingroup$ What is $\theta$? $\endgroup$
    – kodlu
    Jan 2, 2015 at 10:28
  • $\begingroup$ en.wikipedia.org/wiki/… $\endgroup$
    – Turbo
    Jan 2, 2015 at 19:09
  • $\begingroup$ @Turbo you must state it in the question. $\endgroup$ Jan 3, 2015 at 0:20
  • $\begingroup$ Usually captial \Theta is used: $\Theta$. $\endgroup$
    – usul
    Jan 4, 2015 at 23:35

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.