All Questions
Tagged with it.information-theory st.statistics
9 questions
48
votes
2
answers
14k
views
Research situation in the field of Information Geometry
I am now doing an article survey on the field of information geometry started by S.Amari and Barndorff-Nielson. I want to know some research situation in this field.
I have read (4) and parts of (3). ...
- 8,071
40
votes
5
answers
5k
views
"Entropy" proof of Brunn-Minkowski Inequality?
I read in an information theory textbook the Brunn-Minkowski inequality follows from the Entropy Power inequality.
The first one says that if $A,B$ are convex polygons in $\mathbb{R}^d$, then
$$ m(...
- 22.8k
14
votes
1
answer
3k
views
How is the "conformal prediction" conformal?
The question is clarified by Prof.V.Vovk. See his answer below for discussion.
Recently, early works of Gammerman, Vanpnik and Vovk[4] are rediscovered by Wasserman et.al[1] and proposed it as a ...
- 8,071
10
votes
2
answers
2k
views
Convergence of an empirical distribution w.r.t. the Hellinger distance
Let $P$ be a probability distribution on a finite set $\mathcal{X}$ and let $X_1, X_2, \ldots, X_n$ be drawn i.i.d. according to $P$. Define the empirical distribution:
$\hat{P_n}(x) = \frac{1}{n} \...
- 165
5
votes
1
answer
237
views
Rate-Distortion theory: What is the distribution of distortion on an optimal Gaussian encoder?
If we wish to encode a gaussian source, $X\sim\mathcal{N}(0,\sigma^2)$ at rate $R$, then decode it to create an estimate $\hat{X}$, rate-distortion theory tells us that the lowest mean-squared-error ...
4
votes
3
answers
1k
views
Incremental entropy computation
After a quick internet search I found no method for incremental entropy computation.
Question 1
Let $\{x_i\}_{i=1}^n$ and $\{x_i\}_{i=1+n}^{n+m}$ be two samples and let $S_i^j:=\sum_{k=i}^j x_k$. ...
- 161
3
votes
0
answers
494
views
Maximization of a total variation distance subject to another total variation distance in Markov chain
Suppose two dependent random variables $X$ and $V$ from finite alphabets $\mathcal{V}$ and $\mathcal{X}$ with known joint and marginal distributions are given. Let $P_{XV}$ and $P_X$ and $P_V$ are the ...
- 1,109
3
votes
1
answer
737
views
Exponential deconvolution using the first derivative
There is an interesting observation using the first derivative to deconvolve an exponentially modified Gaussian:
The animation is here at terpconnect.umd.edu.
The main idea is that if we have an ...
- 879
2
votes
1
answer
1k
views
Mutual information between continuous and discrete variables from numerical data
I am looking for references/measures to estimate the mutual information between a continuous (C) and discrete (D) variable, given a real-world (i.e. finite sample) data set. C is uniformly distributed ...
- 21