3
$\begingroup$

I'm studying information theory by myself.

I'm confused about that since we already have Huffman code, which is the optimal code method, why are Shannon code and some other code still useful?

I think maybe in some specific industrial area, Shannon code is more useful.

Can someone give me an explanation?

Improve this question
$\endgroup$
1
  • $\begingroup$ Huffman coding assumes accurate data. Shannon assumes data have noise added. $\endgroup$
    – John D. Cook
    Commented Aug 19, 2012 at 23:07

1 Answer 1

Reset to default
4
$\begingroup$

I have to say that I know almost nothing about Shannon coding, which is different from, for example, Shannon-Fano coding. Shannon-Fano coding is in some sense a precursor of Huffman coding, it works in a similar way, and while it does not produce the smallest expected code word lengths, the code word lengths are within one bit of their theoretical ideal (see http://en.wikipedia.org/wiki/Shannon-Fano_coding). (But see Martin Leslie's comment below.) Apparently Shannon-Fano coding is still used, even though admittedly it is hard to see a reason for this.

On the other hand, Huffman coding is not optimal because it codes only characters. You can improve the coding by coding longer words instead. This is done in arithmetic coding. Huffman coding is still useful because it is easier to compute (or you can rely on a table, for instance using the frequency of characters in the English language).

Finally there are many codes that serve a different purpose than Huffman coding: error detecting and error correcting codes, for example. Such codes are very important in various areas, in particular in industrial applications.

Finally, even if the goal is lossless compression, depending on the data, Huffman code might not be suitable: music, images, movies, and so on.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Yes, I mean Shannon-Fano code..Thanks for your answer, very clear! $\endgroup$
    – user18717
    Commented Aug 19, 2012 at 11:24
  • $\begingroup$ "the code word lengths are within one bit of their theoretical ideal": proof? $\endgroup$
    – darij grinberg
    Commented Aug 19, 2012 at 14:55
  • $\begingroup$ darij, I think that sentence in the wikipedia article is false: See cstheory.stackexchange.com/questions/8314/… for an unanswered (as far as I know) question about Shannon-Fano coding. I provide a counterexample (I think) in my question. $\endgroup$
    – Martin Leslie
    Commented Aug 19, 2012 at 18:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .