MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

For each $n \in \mathbb{N}$ let $X_n$ be a random variable taking its values in a finite set $E_n$ with $P(X_n=x_n)>0$ for all $x_n \in E_n$. Say that $X_n$ is asymptotically degenerate if $\min_{x_n \in E_n}\Pr(X_n \neq x_n) \to 0$. Denoting by $h(X)$ the entropy of (the law of) a discrete random variable $X$, assume that $\frac{h(X_n)}{\log k_n}\to0$ with $k_n = \sharp E_n$. Does it imply that $X_n$ is asymptotically degenerate ?

share|cite|improve this question
I must be misunderstanding something. If $k_n\to\infty$ it automatically follows that $\min_{x_n\in E_n} P(X_n=x_n)\to 0$. On the other hand, if $k_n$ stays bounded then the $\log k_n$ denominator is irrelevant, and then indeed it is true that if the entropy goes to 0 then the distribution asymptotically concentrates on a single value. – James Martin Feb 18 '12 at 22:48
Thanks. Sorry, my definition is $\min_{x_n \in E_n}\Pr(X_n \neq x_n) \to 0$, not $\min_{x_n \in E_n}\Pr(X_n=x_n) \to 0$. I'll edit my post and fix that. – Stéphane Laurent Feb 19 '12 at 5:32
up vote 2 down vote accepted

Of course, not. Your "asymptotic degeneracy" condition means that the weight of the maximal atom of the distribution of $X_n$ goes to 1, which is a pretty strong condition. If the sizes of your sets $E_n$ go to infinity, take a sequence $\epsilon_n\to 0$. Then the sequence of measures $\nu_n$ with two atoms of weight $(1-\epsilon_n)/2$ and the remaining mass $\epsilon_n$ equidistributed among the other points of $E_n$ is a counterexample. On the other hand, if the sizes of $E_n$ are uniformly bounded, then $h(X_n)\to 0$ indeed implies asymptotic degeneracy in your sense.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.