There are a number of definitions of entropy floating around that are analogous even if they aren't equivalent. For any outsider they should look and feel nearly the same, for an expert you may be incline to choose one over the other.
- information entropy
- statistical entropy
- topological entropy
Exciting new variants in theoretical physics
Please excuse my over-simplified treatment. Also people are still thinking about what entropy to this day. Please also In Search of a Structure, Part I: On Entropy (Gromov, 2012)
Boltzmann defined "entropy" as the number of microstates of a thermodynamic ensemble. And his example is the asymptotics of the multinomial distribution.
A If $m = p\, n$ the binomial coefficients grow as the entropy: $$ \binom{n}{m} = e^{p \log p + (1-p) \log (1-p)} = \mathrm{exp}\, h(X) $$ where $X$ is the random variable which is $X=1$ with probability $p$ and $X=0$ with probability $1-p$.
B What about just the map $T: x \mapsto 2x \mod 1$ on $[0,1]$. How many does a typical point $x$ have?
$$ 2^n x \equiv a \mod 1 $$ This has $2^n$ solutions, so the entropy is $\frac{1}{n}\log |T^{-n}(x)| = \log 2$.
These are the two basic kinds of entropy (probabilistic, topological) and other definitions clarify this.
See also: Information Theory and Statistical Mechanics (Jaynes)