All Questions
Tagged with it.information-theory fa.functional-analysis
12 questions
1
vote
0
answers
45
views
Existence of optimal entropic weights for empirical modeling
Let $\mathcal{X} = [0,1]^n$ be the input space and $\mathcal{Y} = \{1, ..., n_c\}$ be a discrete output space. Let $D = \{(x_i, y_i)\}_{i=1}^N \subset \mathcal{X} \times \mathcal{Y}$ be a training ...
- 111
1
vote
1
answer
182
views
Inequalities involving entropy: quantum discord and mutual information
My question is inspired by the following paper of Olivier and Żurek but for this question to be self-contained I will recall all the necessary definitions: for a quantum state $\rho$ we define the ...
- 9,330
1
vote
0
answers
65
views
Normalizing constants preserve metric entropy
Suppose $\mathcal{F}=\left\{f\in L^2([a,b]): 0<\underline{c}\leq f\leq\overline{c} \right\}$. Consider the following transformation
$$\tilde{\mathcal{F}} := \left\{\frac{f}{\int f d\mu}: f\in \...
- 11
0
votes
1
answer
158
views
Encoding numbers with relationship into one and back
Given a set of many variables $S=\{x_1,x_2, ...., x_i\}$, and any subset $S'$ of $S$, I need a function $f$ which maps $S'$ to a value $x$ and a function $f'$ which maps $x$ back to set $S'$.
I know ...
11
votes
1
answer
320
views
Conceptual explanation for the appearance of entropy in $\frac{d}{dp}\|x\|_p$
For $x\in \mathbb{R}^d$, an elementary computation yields that
$$\frac{d}{dp}\log \|x\|_p =\frac{1}{p^2}\sum_{i=1}^d \frac{|x_i|^p}{\|x\|_p^p}\log \frac{|x_i|^p}{\|x\|_p^p}=-\frac{1}{p^2}\operatorname{...
- 775
5
votes
1
answer
450
views
Brascamp-Lieb inequalities on the sphere
In the paper [CLL], Carlen, Lieb, and Loss demonstrate a version of the Young inequality on the sphere $S^{N-1}$ in $\mathbb{R}^N$. For positive functions $f_j$ on $[-1,1]$, the following bound holds:...
- 452
9
votes
1
answer
385
views
A Generalized Version of Maximal Correlation and Hypercontractivity of Conditional Expectation Operator
Given a pair of random variables $(X,Y)$ over a product space $\mathcal{X}\times \mathcal{Y}$, the maximal correlation coefficient is defined as
$$\rho_2(X;Y):=\sup\frac{\mathbb{E}[f(X)g(Y)]}{||f||_2||...
- 1,109
40
votes
5
answers
5k
views
"Entropy" proof of Brunn-Minkowski Inequality?
I read in an information theory textbook the Brunn-Minkowski inequality follows from the Entropy Power inequality.
The first one says that if $A,B$ are convex polygons in $\mathbb{R}^d$, then
$$ m(...
- 22.8k
4
votes
1
answer
638
views
Stability in algebraic geometry
Suppose I have a collection of polynomials with multiple variables (more polynomials than variables, say), and I'm given noisy versions the values of these polynomials at a certain unknown point. I ...
- 7,633
5
votes
2
answers
631
views
Proving that a complicated function is eventually concave
I have a function $f:\mathbb{R}^+ \to \mathbb{R}^+$ that I want to prove is eventually concave - i.e. that there exists $\gamma _0 > 0$ such that for every $\gamma>\gamma_0$, $f(\gamma)$ is ...
- 392
8
votes
2
answers
540
views
Maximum entropy priors in infinite dimensional spaces
Is there an extension of maximum entropy probability distributions for function spaces?
For $\mathbb{R}^n$ and discrete spaces, there is much literature about this problem under names such as "non-...
- 1,160
5
votes
1
answer
666
views
Question regarding divergence
Let $E$ be a closed and convex set of distributions on a finite set $A$. Let $P',Q'\notin E$ and let $P^{\star},Q^{\star}$ be their respective estimates in $E$ with respect to the KL-divergence, i.e.,...
- 779