Return to Question

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Let entropy of $\mathcal{A}=\{a_i\}_{i=1}^m$ be given by $$H(\mathcal{A},m)=-\sum_{i=0}^mp_i\log_2p_i.$$

$H(\mathcal{A},m)$ has interesting properties. Example: discussion in Limiting Entropy of deterministic sequences - 1 shows following holds if $k>0$ as $m\in\Bbb N$ increases:

$$a_{i+1}=a_i+\Theta(\log^ka_i)\implies H(\mathcal{A,m})\mbox{ is unbounded}$$ $$a_{i+1}=a_i+\Theta(a_i^{\frac{1}{k}})\implies H(\mathcal{A,m})\mbox{ is bounded}$$

Here symbol $\Theta$ from Landau notations in complexity theory http://en.wikipedia.org/wiki/Big_O_notation#Family_of_Bachmann.E2.80.93Landau_notations.

Is there a physical interpretation of application of Shannon entropy in this situation? Does it mean we should be able to compress $a_i$ in finite number of bits or impossible to do in finite number of bits in total? This is true if $a_i$s could be thought of 'occurring' with probability $p_i$. This is true if there is an interpretation of $a_i$s as random variables. We generate probabilities from fixed integers which seems to make traditional compression interpretation shaky.

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Let entropy of $\mathcal{A}=\{a_i\}_{i=1}^m$ be given by $$H(\mathcal{A},m)=-\sum_{i=0}^mp_i\log_2p_i.$$

$H(\mathcal{A},m)$ has interesting properties. Example: discussion in Limiting Entropy of deterministic sequences - 1 shows following holds if $k>0$ as $m\in\Bbb N$ increases:

$$a_{i+1}=a_i+\Theta(\log^ka_i)\implies H(\mathcal{A,m})\mbox{ is unbounded}$$ $$a_{i+1}=a_i+\Theta(a_i^{\frac{1}{k}})\implies H(\mathcal{A,m})\mbox{ is bounded}$$

Here symbol $\Theta$ from Landau notations in complexity theory http://en.wikipedia.org/wiki/Big_O_notation#Family_of_Bachmann.E2.80.93Landau_notations.

Is there a physical interpretation of application of Shannon entropy in this situation? Does it mean we should be able to compress $a_i$ in finite number of bits or impossible to do in finite number of bits in total? This is true if $a_i$s could be thought of 'occurring' with probability $p_i$. This is true if there is an interpretation of $a_i$s as random variables. We generate probabilities from fixed integers which seems to make traditional compression interpretation shaky.

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Let entropy of $\mathcal{A}=\{a_i\}_{i=1}^m$ be given by $$H(\mathcal{A},m)=-\sum_{i=0}^mp_i\log_2p_i.$$

$H(\mathcal{A},m)$ has interesting properties. Example: discussion in Limiting Entropy of deterministic sequences - 1 shows following holds if $k>0$ as $m\in\Bbb N$ increases:

$$a_{i+1}=a_i+\theta(\log^ka_i)\implies H(\mathcal{A,m})\mbox{ is unbounded}$$ $$a_{i+1}=a_i+\theta(a_i^{\frac{1}{k}})\implies H(\mathcal{A,m})\mbox{ is bounded}$$

Here symbol $\theta$ from Landau notations in complexity theory http://en.wikipedia.org/wiki/Big_O_notation#Family_of_Bachmann.E2.80.93Landau_notations.

Is there a physical interpretation of application of Shannon entropy in this situation? Does it mean we should be able to compress $a_i$ in finite number of bits or impossible to do in finite number of bits in total? This is true if $a_i$s could be thought of 'occurring' with probability $p_i$. This is true if there is an interpretation of $a_i$s as random variables. We generate probabilities from fixed integers which seems to make traditional compression interpretation shaky.

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Let entropy of $\mathcal{A}=\{a_i\}_{i=1}^m$ be given by $$H(\mathcal{A},m)=-\sum_{i=0}^mp_i\log_2p_i.$$

$H(\mathcal{A},m)$ has interesting properties. Example: discussion in Limiting Entropy of deterministic sequences - 1 shows following holds if $k>0$ as $m\in\Bbb N$ increases:

$$a_{i+1}=a_i+\Theta(\log^ka_i)\implies H(\mathcal{A,m})\mbox{ is unbounded}$$ $$a_{i+1}=a_i+\Theta(a_i^{\frac{1}{k}})\implies H(\mathcal{A,m})\mbox{ is bounded}$$

Here symbol $\Theta$ from Landau notations in complexity theory http://en.wikipedia.org/wiki/Big_O_notation#Family_of_Bachmann.E2.80.93Landau_notations.

Is there a physical interpretation of application of Shannon entropy in this situation? Does it mean we should be able to compress $a_i$ in finite number of bits or impossible to do in finite number of bits in total? This is true if $a_i$s could be thought of 'occurring' with probability $p_i$. This is true if there is an interpretation of $a_i$s as random variables. We generate probabilities from fixed integers which seems to make traditional compression interpretation shaky.