For each $n \in \mathbb{N}$ let $X_n$ be a random variable taking its values in a finite set $E_n$ with $P(X_n=x_n)>0$ for all $x_n \in E_n$. Say that $X_n$ is asymptotically degenerate if $\min_{x_n \in E_n}\Pr(X_n \neq x_n) \to 0$. Denoting by $h(X)$ the entropy of (the law of) a discrete random variable $X$, assume that $\frac{h(X_n)}{\log k_n}\to0$ with $k_n = \sharp E_n$. Does it imply that $X_n$ is asymptotically degenerate ?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ I must be misunderstanding something. If $k_n\to\infty$ it automatically follows that $\min_{x_n\in E_n} P(X_n=x_n)\to 0$. On the other hand, if $k_n$ stays bounded then the $\log k_n$ denominator is irrelevant, and then indeed it is true that if the entropy goes to 0 then the distribution asymptotically concentrates on a single value. $\endgroup$– James MartinCommented Feb 18, 2012 at 22:48
-
$\begingroup$ Thanks. Sorry, my definition is $\min_{x_n \in E_n}\Pr(X_n \neq x_n) \to 0$, not $\min_{x_n \in E_n}\Pr(X_n=x_n) \to 0$. I'll edit my post and fix that. $\endgroup$– Stéphane LaurentCommented Feb 19, 2012 at 5:32
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Of course, not. Your "asymptotic degeneracy" condition means that the weight of the maximal atom of the distribution of $X_n$ goes to 1, which is a pretty strong condition. If the sizes of your sets $E_n$ go to infinity, take a sequence $\epsilon_n\to 0$. Then the sequence of measures $\nu_n$ with two atoms of weight $(1-\epsilon_n)/2$ and the remaining mass $\epsilon_n$ equidistributed among the other points of $E_n$ is a counterexample. On the other hand, if the sizes of $E_n$ are uniformly bounded, then $h(X_n)\to 0$ indeed implies asymptotic degeneracy in your sense.
