Search Results
| Search type | Search syntax |
|---|---|
| Tags | [tag] |
| Exact | "words here" |
| Author |
user:1234 user:me (yours) |
| Score |
score:3 (3+) score:0 (none) |
| Answers |
answers:3 (3+) answers:0 (none) isaccepted:yes hasaccepted:no inquestion:1234 |
| Views | views:250 |
| Code | code:"if (foo != bar)" |
| Sections |
title:apples body:"apples oranges" |
| URL | url:"*.example.com" |
| Saves | in:saves |
| Status |
closed:yes duplicate:no migrated:no wiki:no |
| Types |
is:question is:answer |
| Exclude |
-[tag] -apples |
| For more details on advanced search visit our help page | |
Results tagged with it.information-theory
Search options not deleted
user 99418
Theoretical and experimental aspects of information theory and coding theory. This tag covers but is not limited to following branches: information theory, information geometry, optimal transportation theory, coding theory.
1
vote
Accepted
Maximization of information over set of non-injective functions (Equality)
No. Let $Y,Z$ be iid Bernoulli(1/2), and let $X=Y+Z$ mod 2, which induces pairwise independence. The only non-injective deterministic functions $f_Y,f_Z$ are constants, rendering the RHS zero. For th …
- 716
1
vote
Accepted
Smallest $\mathrm{D}(Q\|P)$ given fixed marginals $\mathrm{D}(Q_X\|P_X)$ and $\mathrm{D}(Q_Y...
Here is a related problem: Fix $Q_X$ and $Q_Y$ and ask what coupling $Q$ minimizes $D(Q||P)$. In this case and under mild assumptions, the minimum exists since the set of couplings $\Pi(Q_X,Q_Y)$ is …
- 716
3
votes
Relation between multivariate estimation error and differential entropy
You are missing dimensional constants, but you have the right idea. Namely, it is true that
$$
E|X-\hat{X}(Y)|^2 \geq \frac{m}{2\pi e} e^{2 h(X|Y)/m }.
$$
Like in the 1-dimensional case, this is a co …
- 716
1
vote
The entropy of a partition of unity
In the parlance of information theory, your functional $I$ is a conditional relative entropy, and $H$ is a mutual information. Indeed, $\mu$ and $K$ together define a joint probability measure on $X …
- 716
7
votes
Accepted
Upper-bound on the Fisher-Rao distance between multivariate Gaussian measures by the KL-dive...
Since relative entropy behaves locally like a squared distance, we might expect the squared Fisher-Rao metric to be comparable to the symmetrized KL divergence. This is indeed the case.
Let $d_F$ den …
- 716
1
vote
Accepted
Can information be extracted more precisely using more random trials?
The characterization is given in terms of a so-called auxiliary random variable. It is as explicit of an answer as you'll get, unless you consider very special cases (like jointly Gaussian $X,Y$, or …
- 716
4
votes
"Entropy" proof of Brunn-Minkowski Inequality?
I'm a bit hesitant to resurrect an old post, but as a result of some of the things I worked on recently, I'd like to share a new answer to the question in the title. Hopefully some will find it inte …
- 716