Ok first, the entropy you're talking about is the differential entropy $-\int f(x) \ln f(x) d\mu(x)$. The problem is that $\mu$ is Lebesgue measure. The set of continuous probability distributions is the set of distributions that have a density (i.e. radon-nikodym derivative) wrt Lebesgue measure. As such, if your distribution does NOT have such a density, then there isn't really a meaningful interpretation of the above quantity. Not only that, if your distribution had a density $f$ wrt to some other measure $\nu$, and you plugged $f$ into the above, you'd accidentally be computing the differential entropy of some completely different distribution $f d\mu$, which *is* a continuous distribution. So to answer the question: if you want to deal with non-continuous distributions, you have to tweak your definition of differential entropy. if the only difference you make is to substitute in some well-behaved measure $\nu$, then the derivation will go through the same, but the distribution you get out will be wrt $\nu$, ie not the same as $\phi d\mu$ where $\phi$ is the density of the gaussian. PS a good reference on this stuff is cover&thomas's information theory book, which has a derivation of gaussian being the max (differential) entropy (continuous) distribution with constant variance.