0
$\begingroup$

Since a few days, I try in my research to model / formalize a source of Shannon a little weird, and I can't do it at all. First of all, I explain to you its operating principle and then I describe it to you precisely.

Let a discrete source X and a capacitance channel C, if we define the entropy of the source (in other words the density of the information, its "richness") by H(X), so if we have H(X)>C, Shannon's conditions are not verified and it's not certain that the noisy channel can capture information without error. So in this case, this curious source starts to weight, to give a notion of importance, all the information it emits, so that information of too low weight is "lost" and thus to decrease its entropy H(X) until it returns to the conditions of Shannon and to obtain H(X) < C and thus to transmit the information.

Here is a detailed description of this source:

  1. Source S transmits the information (A1, A2, A3, A4, A5, ..., An) through a noisy channel of capacitance C.

  2. The source S is variable, that is to say that it transmits (A1, A2, A3, A4, A5, ... Aj) with j which varies randomly from 1 to n.

  3. There exists a p such that the entropy H of S is < C when it transmits (A1, A2, A3, A4, A5, ..., Aj) with j < or=p but > C with j>p.

  4. Since the information A1 and An are less important, they no longer transmit it when j>p so that its entropy H is again lower than C and the information passes. (If you have to fire n-p informations to reduce the entropy H below C, we can say that there are n-p informations less important than the source S can potentially transfer).

I tried to represent this system by associating to each Ai a probability pi etc., I tried to tweak the formula of Shannon's entropy to express that I I'm moving out the information A1 and An but in vain, I didn't succeed in writing the "variable" aspect of the source S, I also failed to write mathematically the notion of importance for A1 and An... in short I struggle completely and I bug on almost all the points of my problem to write it correctly.

I need to write down the properties of this source properly to do my calculations, if ever someone can help me, I really thank you. If Shannon's probabilities are not enough to write this source, and we need Kolmogorov's formalism, I am open :)

PS : sorry for my english I am french

Best regards,

Lulu2612

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ This is quite incomprehensible, it's not an English problem per se. Do you mean $A_i$ has probability $p_i$? so you have an $n$ symbol alphabet $\{A_1,\ldots,A_n\}$? (you should use Latex for equations). What is the source transmitting? A sequence $X_k,$ for $k\geq 1,$ so that $P[X_k=A_i]=p_i$? $\endgroup$
    – kodlu
    Commented Nov 28, 2018 at 11:00
  • 2
    $\begingroup$ As for the less important, more important part, and leaving symbols out, it is really difficult to understand what you mean. Please look at the information theory course page at web.stanford.edu/class/ee376a/reading.html and rewrite what you want in the standard terminology used in this course. $\endgroup$
    – kodlu
    Commented Nov 28, 2018 at 11:02
  • $\begingroup$ Yes, Ai has probability pi, and you write well about the n symbols and about the sequence Xk. But the source is variable. That means that it transmits (A1, A2, A3, A4, A5, ... Aj) with j≤n. And when its entropy H(X)>C, it doesn't transfer as much information as it needs to reduce its entropy. That is to mean, it self-manages entropy to maintain H(X)<C. For this, each Ai is assigned a "coefficient of importance" λi. And the source no longer transfers information Ai that has a small λi. If you can help me to describe this source with the standard terminology thank you very much $\endgroup$
    – lulu2612
    Commented Nov 28, 2018 at 16:52

1 Answer 1

Reset to default
1
$\begingroup$

The concept of weighted entropy with weight function $\varphi$ defined as

$$ H_\varphi = -\sum_i \phi(A_i) p(A_i) \log p(A_i) $$

is not so new. However, this recent reference seems to give a good discussion.

https://arxiv.org/abs/1710.10798

If I understand your problem correctly you need to find weights that give you a weighted entropy below the channel capacity and correspond to your evaluation of the relative importance of the different symbols. This is an optimisation problem.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Wao thank you very very much ! You understand exactly what I need ! I am going to read this article carefully ! $\endgroup$
    – lulu2612
    Commented Nov 29, 2018 at 7:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .