The entropy tag has no wiki summary.

**1**

vote

**1**answer

362 views

### Calculate channel capacity of general channel under constraint

Hi!
Given a conditional distribution $P_{Y|X}$ I'd like to find the prior distribution $P_X$ that maximizes the mutual information $I(X;Y)$ with $P_Y(y)=\int P_{Y|X}(y|x)P_X(x)\text{dx}$ (this ...

**11**

votes

**0**answers

168 views

### Entropy in elimination theory, or a brief remark by Gelfand-Kapranov-Zelevinsky

In the introduction to their book "Discriminants, resultants and multidimensional determinant", the authors state a very intriguing observation concerning the coefficients of monomials appearing in ...

**5**

votes

**0**answers

199 views

### Maximizing Renyi entropy for a certain channel

The channel under consideration is $T = A + B$, where $A$ and $B$ take on values in $\{0, 1\}$ according to a probability mass function. Let (joint) random vector $(A_1, A_2,\ldots, A_n)$ be denoted ...

**4**

votes

**0**answers

78 views

### $q$-deformations of fundamental equation of information and entropies

Classical information theory: fundamental equation of information
In classical information theory, the information $I(A)$ of an event $A$ (any element of the $\sigma$-algebra $\mathcal F$ of a ...

**3**

votes

**0**answers

76 views

### How many samples to accurately estimate the entropy

Consider a discrete random variable $N$ with range $\{1,\dots,1000\}$, say. How many samples do you need to take to be sure that you can, at least in principle, compute the entropy Shannon entropy ...

**3**

votes

**0**answers

118 views

### On the volume entropy of negatively curved manifolds

Let $X$ be the universal cover of a closed negatively curved Riemannian manifold. Let $x_0\in X$ be a base point, $S$ be the unit sphere in $T_{x_0}X$ and $\exp:T_{x_0}X\rightarrow X$ be the ...

**3**

votes

**0**answers

86 views

### Is there a known generalization of the Schmidt decomposition based on a maximal set of “locally orthogonal” vectors?

I came across the following unusual generalization of the Schmidt decomposition in my work, which I describe below. I would like to know if this structure has been studied before so I can read more ...

**3**

votes

**0**answers

615 views

### Entropy conjecture for flows

The entropy conjecture for diffeomorphisms (see for example this paper) asserts that for diffeomorphisms of manifolds, the log of the spectral radius of the actions of the diffeomorphism on the ...

**2**

votes

**0**answers

65 views

### Concentration bound in high min entropy distribution

Let $(X_{1},\dots,X_{m})$ be joint distribution on $\{0,1\}^{m}$ with that $H_{\infty}(X_{1},\cdots,X_{m})\geq m-r$, where $H_{\infty}$ means min-entropy.
Let $P_{1},...,P_{n}\subseteq [m]$ be sets ...

**2**

votes

**0**answers

41 views

### minimal entropy approximation of a discrete random variable

Let $X$ be a $\mathbb{N}$-valued random variable. Define
$$
H^\epsilon_n(X) = \inf_f H(f(X))
$$
where $f$ runs over all functions $\mathbb{N} \to \mathbb{N}$ such that $\Pr(f(X)\neq X)<\epsilon$ ...

**2**

votes

**0**answers

44 views

### Sequence transformations that are entropy invariant

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Define entropy of $\mathcal{A}=\{a_i\}_{i=1}^m$
by ...

**2**

votes

**0**answers

122 views

### Interpretation of Shannon Entropy Application

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Let entropy of $\mathcal{A}=\{a_i\}_{i=1}^m$ be given
by ...

**2**

votes

**0**answers

31 views

### largest size for a randomness extractor

I am not so expert in theoretical computer science, so sorry if the question is trivial, i just could not find it in literature.
Suppose we have a source $X$ with min-entropy $\ell$, the randomness ...

**2**

votes

**0**answers

51 views

### Private Randomness extractor

Suppose we are given two random variables $X$ and $Y$ with fixed marginal and joint distribution. What is the maximum randomness that we can extract from $Y$ that is independent from $X$, that is, if ...

**2**

votes

**0**answers

146 views

### metric entropy for Lipschitz functions

Suppose $(X,d)$ is a metric space of unit diameer and let $F$ be the collection of all $1$-Lipschitz functions mapping $X$ to $[-1,1]$, equipped with the sup-norm $||\cdot||_\infty$.
I am interested ...

**2**

votes

**0**answers

130 views

### Reference for and Properties of the $alpha$-entropy

Let $T : X \to X$ be a continuous map on, say, a compact metric space $X$. Let $\mu$ be an invariant borel measure. Under suitable conditions, a result of Brin and Katok states that $\mu$-almost ...

**2**

votes

**0**answers

105 views

### order of convergence of the conditional entropy (3)

I'm sorry for having open two questions which have been solved by elementary counter-examples provided by @AnthonyQuas. Actually I'm not an expert in information theory and I expected that a positive ...

**2**

votes

**0**answers

122 views

### A.G. Vitushkin's “Easily representable families of functions” - can it be generalized?

Background
In his monograph "Estimation of the complexity of the tabulation problem" (translated into English as "Theory of the Transmission and Processing of Information") Vitushkin studies ...

**2**

votes

**0**answers

296 views

### Does this inequality of negative relative entropy and quantum relative entropy hold?

Hello, everyone!
Question
I have a question about the relationship between general relative entropy and general quantum relative entropy: Given a unit vector $|i\rangle$ and two Hermitian matrices ...

**1**

vote

**0**answers

42 views

### Limiting Entropy of deterministic sequences - 2

Consider a collection of increasing positive integers $\{a_i\}_{i=1}^m$ and the distribution $p_i=\frac{a_i}{\sum_{i=1}^ma_i}$. Given $\{a_i\}_{i=1}^m$, let $\mathcal{P}_a$ be limiting distribution ...

**1**

vote

**0**answers

37 views

### Characterization of the optimal solution in relative entropy minimization

The following optimization problem is related to relative entropy and to the limit of the iterative proportional fitting procedure.
For $1 \leq i,j \leq n$ and fixed $w_{ij} \geq 0$, and fixed $a_i, ...

**1**

vote

**0**answers

138 views

### Entropy of factors of Bernoulli schemes

Let $X$ be a Bernoulli scheme. A factor $\psi: X \to Y$ is finitary if for almost every $x \in X$ there exist integers $m \leq n$ such that the zero coordinates of $\psi(x)$ and $\psi(x')$ agree for ...

**1**

vote

**0**answers

247 views

### Incremental computation of a conditional entropy

Is it possible to compute a conditional entropy (see the two following formulas) in an incremental manner ? That is, the sets C and K are not fix: each time we have a new element c, set K may increase ...

**1**

vote

**0**answers

85 views

### Median entropy to observe evolution of system?

Hello,
I am studying a dynamical system that takes as an initial condition a list. I want to analyze the evolution of Shannon's entropy in this system. I know the maximum entropy (50) and the minimum ...

**0**

votes

**0**answers

35 views

### minimal entropy approximation of a truncated discrete measure

Consider a measure $\mu$ on $\mathbb{N}$ given by the sequence $(\mu(n))_{n \geq 0}$ with $\mu(0)>0$. For example $\mu(n)=n^2+1$ on the figure below.
For each $n$, let $X_n \sim \mu(\cdot \mid ...

**0**

votes

**0**answers

265 views

### Calculating entropy of adjacency matrix using eigenvalue decomposition?

How to calculate entropy using the eigenvalues when the eigenvalues are negative?
Is there a simple relation between the entropy of a matrix and its characteristic polynomial?