# Questions tagged [it.information-theory]

Theoretical and experimental aspects of information theory and coding theory. This tag covers but is not limited to following branches: information theory, information geometry, optimal transportation theory, coding theory.

498 questions
Filter by
Sorted by
Tagged with
18 views

### Maximum mutual information of random unitary transformation

Let $\mathbf{U}$ and $\mathbf{V}$ be random unitary matrices independent of random input vector $\mathbf{x}$. Moreover, $\mathbf{z}$ be random iid complex Gaussian vector with zero mean and unit ...
107 views

192 views

### Maximum mutual information of a matrix representation

Let $\mathbf{Z}$ be a $m\times n$ matrix with zero-mean unit-variance i.i.d complex Gaussian entries. What is the maximum value of mutual information $I(\mathbf{X}_1,\mathbf{X}_2;\mathbf{Y})$ such ...
60 views

128 views

102 views

### Information theory for uncountably infinite-dimensional continuous random variable

I'm exploring the possibility to apply information theory on an uncountably infinite-dimensional scenario. I found the concept of generalized entropy for continuous random variables defined on finite-...
148 views

### Is there a difference between using nats and bits to express entropy?

It seems to me like questions involving decimal vs binary representations of some number are not particularly interesting: for instance $\pi$ or $\sqrt{2}$ are conjectured to be normal in every base, ...
170 views

### Books to develop a deep understanding of Algorithmic Information Theory?

I'm mathematical physicist working with hydrodynamics modelling. Recently, I had to turn to modelling of flows with particles and some questions I have I think are related to Algorithmic Information ...
295 views

### Identifying a subset with as few tests as possible

Informal description: You are given a set of $n$ blood samples, each having probability $p$ of being infected with a disease. Your goal is to determine the set $P$ of infected samples with as few ...
### $\varepsilon$-net of a $d$-dimensional unit ball formed by power set of $V = \{+1, 0 -1\}^d$
I have a set of $d$-dimensional vectors $V = \{+1, 0, -1\}^d$. Then $P(V)$ constitutes the power set of $V$. I now construct a set of unit vectors $V_{\mathrm{sum}}$ from the power set $P(V)$ such ...
Write $n$ iid draws of $(x,y)$ as $(x^n, y^n)$. Fix $R\in (0,H(x))$. What is the min of $n^{-1}H(y^n|f(x^n))$ over $f$ with $H(f(x^n))\leq nR$, taking $n\to \infty$?