Entropy equals zero?

Question

Imagine you have a shift invariant ($\sigma$-invariant) probability measure $\eta$ in the Bernoulli space $\{0,1\}^{\mathbb{N}}$. Define

$\mathcal{P} = \{[0],[1]\}$;

$\mathcal{P}^{n} = \mathcal{P}\vee...\vee \sigma^{-n+1}(\mathcal{P})$ (cylinders of length $n$);

$k_{n} = \#\{P \in \mathcal{P}^{n}:\eta(P)\leq\frac{1}{2^{n}}\}$.

For all $n$, enumerate all the cylinders of length $n$ in increasing order with respect the weight that the measure $\eta$ gives to the cylinders. Say $\{P^{n}_{1},...,P^{n}_{2^{n}}\}$. Assume now a simple condition for $\eta$:

$(\sum_{j=1}^{k_{n}}\eta(P^{n}_{j}) +\frac{2^{n}-k_{n}}{2^{n}}) \rightarrow_{n} 0$

Is it true that $h_{\eta}(\sigma)=0$?

Does anyone have any idea about how to prove this? Or, of course, an example the contradicts it?

I could proof that if the entropy is zero, that condition above is satisfied. What I would like to prove is this:

$h_{\eta}(\sigma)=0$ if and only if $(\sum_{j=1}^{k_{n}}\eta(P^{n}_{j}) +\frac{2^{n}-k_{n}}{2^{n}}) \rightarrow_{n} 0$

Thanks for your attention

Answer 1

EDIT - The answer below deals with an ergodic m.p.s

As this question got up-voted, I've decided to fuly write a solution, based on the sketch I've made in the comments.

Fix some $\varepsilon>0$ small, and $n \gg _\varepsilon 0$, and denote by $C_{n}$ to be the cylinders of length $n$. Let $h=h_{\mu}(\sigma)$ be the metric (Kolmogorov-Sinai) entropy of the system $(\Sigma,\sigma,\mu)$. Moreover, assume $\mu$ is $\sigma$-ergodic measure!

By the Shannon-McMillan-Breiman theorem, there exists a partition of $C_n$ into two sets - $G_{n}, B_{n}$ where $G_{n}$ are the ''good'' cylinders, namely for every $g\in G_{n}$ we have $\mu(g_{n})\approx 2^{-n(h\pm\varepsilon)}$. Now for the ''bad'' cylinders, we have that $\sum_{b\in B_{n}}\mu(b) < \varepsilon$.

Moreover, define the set $S_{n}$ to be the ''small'' cylinders, namely $s\in S_{n}$ iff $\mu(s)\leq 2^{-n}$.

From now on, assume $h<1$.

We see that $G_{n} \cap S_{n} = \emptyset$, as $\mu(g)\geq 2^{-n(h+\varepsilon)}>2^{-n}$ for suitably chosen $\varepsilon$ and $g\in G_{n}$, hence $\sum_{s\in S_{n}} \mu(s) \leq \sum_{s\in B_{n}}\mu(s) <\varepsilon$.

Now we want to estimate $2^{n} - |S_{n}|$, as $|G_{n}|\leq 2^{n} - |S_{n}|$, we first bound $|G_{n}|$.

By a crude packing bound we get $|G_{n}|\approx (1-\varepsilon)2^{n(h\pm\varepsilon)}$.

Now we need to estimate $B_{n}\setminus S_{n}$. The atoms in $B_{n}$ are of two types - large atoms (more than a typical one of the atoms in the good set), and small atoms which are not tiny (namely between $2^{-n}$ and $2^{-n(h+\varepsilon)}$). The number of the larger ones is at-most $\varepsilon 2^{n(h+\varepsilon)}$, and the number of the smaller ones is at-most $\varepsilon 2^{n}$, by a simple union bound and again a volume packing argument.

Therefore, the total number of those atoms is $\leq \varepsilon 2^{n}+o(2^{n})$, which translates to $\frac{2^{n}-|S_{n}|}{2^{n}} \lesssim \varepsilon$.

Hence $\lim_{n} \sum_{s\in S_{n}}\mu(s)+\frac{2^{n}-|S_{n}|}{2^{n}} =0 $ for any measure with $h<1$.

Notice that for $h=1$ (recall I normalize entropy to $log_{2}$ basis), you get simply the Lebesgue measure, as we have uniqueness of measure of maximal entropy in this system, and in that case, $\sum_{s\in S_{n}}\mu(s)+\frac{2^{n}-|S_{n}|}{2^{n}} =1$ by a simple computation.

Answer 2

I wanted to put this as as comment but I don't have enough reputation for that so I am sorry it is in the wrong place. Anyways, the counting argument described by Asaf can show that if $h_\eta (\sigma)<\log 2$ then in your notation $\sum_{n=1}^{k_n} \eta \left(P_j ^n\right)\to 0$. Indeed, in that case, let $h_\eta (\sigma)<\rho<\log 2$ and denote by $G^n\subset \mathcal{P}^n$ be the collection of all sets of measure greater or equal to $e^{-n\rho}$ (these sets are the good sets in Asaf's reply). By the SMB Theorem $$\eta \left(\cup_{P\in G_n}P\right)\to 1,\ as\ n\to\infty.$$ The conclusion follows since for all $1\leq k\leq k_n$, $P_j ^n\notin G^n$. What I don't see immediately is why entropy $0$ implies $\frac{2^n-k_n}{2^n}\to 0$. It is true that by a counting argument (since $\eta(X)=1$) there are at most $e^{n\rho}=o\left(2^n\right)$ sets in $G^n$ but I don't see why shouldn't there be more than $C2^n$ sets in $\mathcal{P}^n$ of measure greater or equal to $2^{-{(n-1)}}$?