Skip to main content

All Questions

Filter by
Sorted by
Tagged with
9 votes
2 answers
1k views

Expected centered entropy of the binomial distribution

In short, the function I am interested in is the following: $$I_n(p) = \sum_{k=0}^n \binom{n}{k} p^k (1-p)^{n-k} \left[h(p) - h\left(\tfrac{k}{n}\right)\right],$$ where $h(x) \triangleq -x \log x - (1-...
TMM's user avatar
  • 733
3 votes
3 answers
392 views

Asymptotic value of the Shannon entropy

I would like to evaluate the asymptotic value of the following sum: $$f(N)=\frac{1}{2^N}\sum_{n=0}^{N} \binom{N}{n} \log_{2} \binom{N}{n}$$ This is related to the computation of the Shannon entropy. ...
Ryan's user avatar
  • 31
1 vote
2 answers
1k views

Approximation of the sum involving binary entropy function

Given the following sum: $S(n) = \sum_{i=1}^{n} \frac{1}{(1-\operatorname{H}(p))^i}$ where $H$ is the binary entropy function defined as: $\operatorname{H}(p) = -p\log p - (1-p)\log (1-p) $. Let $...
Kelvin Lee's user avatar