The physicist Edwin T. Jaynes wrote a lot about entropy and its uses in physics and probability theory:
http://bayes.wustl.edu/etj/node1.html some published works
http://bayes.wustl.edu/etj/node2.html some unpublished works
He insisted that the two are very much related. In one of his works he says he had a conversation with Eugene Wigner in the 1950s in which he told Wigner that physical entropy is a measure of information and Wigner thought that was absurd, because the information one person possesses differs from that of another, whereas entropy can be measured with thermometers and calorimeters. Jaynes says he didn't know at the time how to explain to Wigner why that was wrong, but figured it out much later. As nearly as I understand it at this moment, Jaynes thought entropy measures the amount of information delivered by those thermometers and calorimeters.
Jaynes has a sort of cult following, many members of which are professors in the physical sciences who learned everything they know about probability from Jaynes, and they can sometimes come across a bit like religious fanatics.
OK, a short answer: The entropy in a discrete probability distribution is $\displaystyle\sum_k -p_k\log p_k$, where $\{p_k\}$ are the probabilities assigned to atoms in the probability space. The base of the logarithm can be any number $>1$ (or maybe $<1$ in some cases? I haven't thought about that.). There's also relative entropy of one probablity measure $p$ with respect to another, $q$, given by $\displaystyle\sum_k p_k\log(p_k/q_k)$ and cross entropy given by $\displaystyle\sum_k -p_k\log q_k$.
Via Google, you might be able to find notes taken by scribes in Gian-Carlo Rota's probability course at MIT. Somewhere among those, he says the resemblance between entropy of discrete probability distributions (as defined above) and entropy of continuous distributions (done similarly with integrals.) is only superficial.