I have a question about increasing mutual information for the binary channel. Assuming there is an independently $K$ dimensional binary source signal denoted by $X=[X_1, X_2, \cdots, X_K]$, a parallel noisy channel $N=[N_1, N_2, \cdots, N_K]$, and a received signal $Z=[Z_1, Z_2, \cdots, Z_K]$. $\Pr\{X_i = 1\} = \alpha_i$, and $\Pr\{N_i = 1\} = \beta_i$. The noise is added on the signal, i.e., $Z_i = X_i + N_i \, \text{(mod 2)}$.

Since the signals in each channel are independent, the mutual information between $X$ and $Z$ can be computed as \begin{align} I(X;Z) &= \sum_{i=1}^K I(X_i;Z_i) \\ &= \sum_{i=1}^K (H(Z_i)-H(\beta_i)). \end{align}

Now we want to apply an addition operator on $X$ to increase the mutual information. For example \begin{align} Y_1 &= \sum_{i=1}^K a_{1i} \cdot X_i \\ Y_2 &= \sum_{i=1}^K a_{2i} \cdot X_i \\ Y_j &= \sum_{i=1}^K a_{ji} \cdot X_i, \end{align} where $a_{ji} \in \{0,1\}$. The signal $Y$ is transmitted through the noise channel, i.e., $Z_i = Y_i + N_i \, \text{(mod 2)}$. Thus, we construct a Markov chain $X \rightarrow Y \rightarrow Z$, and the variables in $Y$ are not independent any more.

My question is how to find an optimal addition operator, i.e., the optimal set $\{a_{ji}\}$, such that the mutual information $I(X;Z)$ can be maximized?

  • $\begingroup$ What is the relationship between $Y_i$ and $Z_k$ for $1\leq i,k\leq K$? Is there a channel there? $\endgroup$ – kodlu Feb 22 '16 at 7:07
  • $\begingroup$ $Z_i = Y_i + N_i$. The signal $Y_i$ is only transmitted through the $i$th channel. $\endgroup$ – Schafer Feb 22 '16 at 7:28
  • $\begingroup$ thanks. Are the sums in the expressions for $Y_j$ modulo 2? or over the integers? $\endgroup$ – kodlu Feb 22 '16 at 13:00
  • $\begingroup$ $a_{ji}$ is a function of $X_i$? $\endgroup$ – JonMark Perry Feb 22 '16 at 15:41
  • $\begingroup$ Yes, all the add operations are modulo 2. So 1+1=0, 0+1=1, 0+0=0. $\endgroup$ – Schafer Feb 22 '16 at 21:42

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.