The entropy tag has no wiki summary.

**1**

vote

**1**answer

89 views

### Entropy inequality

Let $P,Q$ be probabilities on a finite set $A$ with $Q(a)\gt 0$, for all $a\in A$, and let $H(P),H(Q)$ denote the entropy and Kullback-Leibler distance respectively.
Is it always true that
...

**1**

vote

**1**answer

400 views

### Calculate channel capacity of general channel under constraint

Hi!
Given a conditional distribution $P_{Y|X}$ I'd like to find the prior distribution $P_X$ that maximizes the mutual information $I(X;Y)$ with $P_Y(y)=\int P_{Y|X}(y|x)P_X(x)\text{dx}$ (this ...

**11**

votes

**0**answers

173 views

### Entropy in elimination theory, or a brief remark by Gelfand-Kapranov-Zelevinsky

In the introduction to their book "Discriminants, resultants and multidimensional determinant", the authors state a very intriguing observation concerning the coefficients of monomials appearing in ...

**5**

votes

**0**answers

202 views

### Maximizing Renyi entropy for a certain channel

The channel under consideration is $T = A + B$, where $A$ and $B$ take on values in $\{0, 1\}$ according to a probability mass function. Let (joint) random vector $(A_1, A_2,\ldots, A_n)$ be denoted ...

**4**

votes

**0**answers

79 views

### $q$-deformations of fundamental equation of information and entropies

Classical information theory: fundamental equation of information
In classical information theory, the information $I(A)$ of an event $A$ (any element of the $\sigma$-algebra $\mathcal F$ of a ...

**3**

votes

**0**answers

79 views

### How many samples to accurately estimate the entropy

Consider a discrete random variable $N$ with range $\{1,\dots,1000\}$, say. How many samples do you need to take to be sure that you can, at least in principle, compute the entropy Shannon entropy ...

**3**

votes

**0**answers

122 views

### On the volume entropy of negatively curved manifolds

Let $X$ be the universal cover of a closed negatively curved Riemannian manifold. Let $x_0\in X$ be a base point, $S$ be the unit sphere in $T_{x_0}X$ and $\exp:T_{x_0}X\rightarrow X$ be the ...

**3**

votes

**0**answers

88 views

### Is there a known generalization of the Schmidt decomposition based on a maximal set of “locally orthogonal” vectors?

I came across the following unusual generalization of the Schmidt decomposition in my work, which I describe below. I would like to know if this structure has been studied before so I can read more ...

**3**

votes

**0**answers

620 views

### Entropy conjecture for flows

The entropy conjecture for diffeomorphisms (see for example this paper) asserts that for diffeomorphisms of manifolds, the log of the spectral radius of the actions of the diffeomorphism on the ...

**2**

votes

**0**answers

72 views

### Concentration bound in high min entropy distribution

Let $(X_{1},\dots,X_{m})$ be joint distribution on $\{0,1\}^{m}$ with that $H_{\infty}(X_{1},\cdots,X_{m})\geq m-r$, where $H_{\infty}$ means min-entropy.
Let $P_{1},...,P_{n}\subseteq [m]$ be sets ...

**2**

votes

**0**answers

48 views

### minimal entropy approximation of a discrete random variable

Let $X$ be a $\mathbb{N}$-valued random variable. Define
$$
H^\epsilon_n(X) = \inf_f H(f(X))
$$
where $f$ runs over all functions $\mathbb{N} \to \mathbb{N}$ such that $\Pr(f(X)\neq X)<\epsilon$ ...

**2**

votes

**0**answers

46 views

### Sequence transformations that are entropy invariant

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Define entropy of $\mathcal{A}=\{a_i\}_{i=1}^m$
by ...

**2**

votes

**0**answers

136 views

### Interpretation of Shannon Entropy Application

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Let entropy of $\mathcal{A}=\{a_i\}_{i=1}^m$ be given
by ...

**2**

votes

**0**answers

32 views

### largest size for a randomness extractor

I am not so expert in theoretical computer science, so sorry if the question is trivial, i just could not find it in literature.
Suppose we have a source $X$ with min-entropy $\ell$, the randomness ...

**2**

votes

**0**answers

51 views

### Private Randomness extractor

Suppose we are given two random variables $X$ and $Y$ with fixed marginal and joint distribution. What is the maximum randomness that we can extract from $Y$ that is independent from $X$, that is, if ...

**2**

votes

**0**answers

160 views

### metric entropy for Lipschitz functions

Suppose $(X,d)$ is a metric space of unit diameer and let $F$ be the collection of all $1$-Lipschitz functions mapping $X$ to $[-1,1]$, equipped with the sup-norm $||\cdot||_\infty$.
I am interested ...

**2**

votes

**0**answers

132 views

### Reference for and Properties of the $alpha$-entropy

Let $T : X \to X$ be a continuous map on, say, a compact metric space $X$. Let $\mu$ be an invariant borel measure. Under suitable conditions, a result of Brin and Katok states that $\mu$-almost ...

**2**

votes

**0**answers

106 views

### order of convergence of the conditional entropy (3)

I'm sorry for having open two questions which have been solved by elementary counter-examples provided by @AnthonyQuas. Actually I'm not an expert in information theory and I expected that a positive ...

**2**

votes

**0**answers

124 views

### A.G. Vitushkin's “Easily representable families of functions” - can it be generalized?

Background
In his monograph "Estimation of the complexity of the tabulation problem" (translated into English as "Theory of the Transmission and Processing of Information") Vitushkin studies ...

**2**

votes

**0**answers

299 views

### Does this inequality of negative relative entropy and quantum relative entropy hold?

Hello, everyone!
Question
I have a question about the relationship between general relative entropy and general quantum relative entropy: Given a unit vector $|i\rangle$ and two Hermitian matrices ...

**1**

vote

**0**answers

97 views

### Estimates of entropy of functional spaces

Let $M^n$ be a compact $n$-dimensional manifold. For $k\geq 0$ let us denote by $C^k(M)$ the Banach space of $k$ times continuously differentiable functions, and $B_{C^k}$ denote the unit ball of it.
...

**1**

vote

**0**answers

42 views

### Limiting Entropy of deterministic sequences - 2

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Given $\{a_i\}_{i=1}^m$, let $\mathcal{P}_a$ be limiting distribution ...

**1**

vote

**0**answers

43 views

### Characterization of the optimal solution in relative entropy minimization

The following optimization problem is related to relative entropy and to the limit of the iterative proportional fitting procedure.
For $1 \leq i,j \leq n$ and fixed $w_{ij} \geq 0$, and fixed $a_i, ...

**1**

vote

**0**answers

141 views

### Entropy of factors of Bernoulli schemes

Let $X$ be a Bernoulli scheme. A factor $\psi: X \to Y$ is finitary if for almost every $x \in X$ there exist integers $m \leq n$ such that the zero coordinates of $\psi(x)$ and $\psi(x')$ agree for ...

**1**

vote

**0**answers

247 views

### Incremental computation of a conditional entropy

Is it possible to compute a conditional entropy (see the two following formulas) in an incremental manner ? That is, the sets C and K are not fix: each time we have a new element c, set K may increase ...

**1**

vote

**0**answers

85 views

### Median entropy to observe evolution of system?

Hello,
I am studying a dynamical system that takes as an initial condition a list. I want to analyze the evolution of Shannon's entropy in this system. I know the maximum entropy (50) and the minimum ...

**0**

votes

**0**answers

39 views

### minimal entropy approximation of a truncated discrete measure

Consider a measure $\mu$ on $\mathbb{N}$ given by the sequence $(\mu(n))_{n \geq 0}$ with $\mu(0)>0$. For example $\mu(n)=n^2+1$ on the figure below.
For each $n$, let $X_n \sim \mu(\cdot \mid ...

**0**

votes

**0**answers

286 views

### Calculating entropy of adjacency matrix using eigenvalue decomposition?

How to calculate entropy using the eigenvalues when the eigenvalues are negative?
Is there a simple relation between the entropy of a matrix and its characteristic polynomial?