# Questions tagged [entropy]

The entropy tag has no usage guidance.

234
questions

**1**

vote

**0**answers

56 views

### A result of the covering number

Suppose $\mathcal{F} = \{f_x : x \in \mathbb{R}^d \}$ and each $f_x$ shares the same law $P$. If $\mathcal{F}$ is a class of uniformly bounded functions satisfying $L_r$-continuity, i.e. $\forall f \...

**4**

votes

**1**answer

250 views

### Are these topological sequence entropy definition equivalent?

I am working on Möbius disjointness for models of topological dynamic systems. In that purpose, I try to understand the notion of topological entropy. We know, for a t.d.s $(X,T)$ that it is defined ...

**1**

vote

**1**answer

50 views

### Entropy condition for quasi-linear evolution equations

Let's consider the problem:
$$
\partial_t u + \partial_x(f(u)) = 0, (x,t)\in \mathbb R \times \mathbb R^+.\\
u|_{t=0}=u_0.
$$
I have seen three formulations for the entropy condition of this equation. ...

**1**

vote

**0**answers

55 views

### Discrete Orlicz space estimate

We consider the discrete LlogL space of sequences $x=(x_i)$ such that
$$\Vert x\Vert_{LL}:=\sum_i \vert x_i \log(x_i)\vert <\infty.$$
Let $x=(x_i)$ and $y=(y_i)$ two sequences in the above LlogL ...

**2**

votes

**0**answers

26 views

### Maximum entropy distribution in the hyperbolic plane with given “mean” and “variance”

On the Euclidean sphere, by defining suitable analogs of the mean and variance in terms of the first circular moment, it can be shown that the von Mises distribution is the maximum entropy ...

**2**

votes

**1**answer

125 views

### Is total variation distance of normalized sum of random variables to Gaussian monotonic decreasing?

Let $X_1, X_2, \ldots$ be independent and identically distributed random variables with mean $0$ and variance $1$ and let $S_n = (X_1 + \cdots + X_n)/\sqrt{n}$ to be their normalized sum. Define $D_n$ ...

**0**

votes

**1**answer

110 views

### Existence of sequence of distributions

This question concerns distributions $\mu$ over the naturals $\mathbb{N}=\{1,2,\ldots\}$. For $q\ge1$, let us define the $q$th moment of entropy:
$$
H_q(\mu)=\sum_{i=1}^\infty \mu(i)|\log\mu(i)|^q,
$$
...

**1**

vote

**0**answers

28 views

### Marginal distribution of $I$-projection

I am reading this paper by Csiszar. Given a probability measure $R$ and a convex subset $\mathcal{E}$ of probability distributions, it defines ‘I-projection of R on $\mathcal{E}$’ (provided there ...

**8**

votes

**1**answer

299 views

### Relative Entropy and p-norm

I asked this question on StackExchange but could not get any answer, therefore, I am posting it here.
I am currently reading the book "A Dynamical Approach to Random Matrix Theory". The ...

**0**

votes

**1**answer

170 views

### Integrability of $\int \log(f(x)) f(x) dx$ for a probability density function $f$

I am looking for weak conditions when a probability density function $f$ on $\mathbb{R}^d$ has a finite integral
$$
\int_{\mathbb{R}^d} \log(f(x)) f(x) dx.
$$
Any references would be appreciated.

**0**

votes

**0**answers

21 views

### Proof of property for Fiedland entropy

I am working with Friedland entropy and there is a proof I cannot figure out how to do.
Friedland entropy is defined for $\mathbb{Z}^k$ continuos actions $\mathcal{T}$ on a topological metric space $X$...

**5**

votes

**3**answers

344 views

### Is there a quantum analog of Kolmogorov Complexity?

Kolmogorov Complexity (interpreted in terms of shortest program computing a string) and Shannon Entropy are quite similar.
Since there is a quantum entropy is it reasonable to ask if there is quantum ...

**8**

votes

**1**answer

268 views

### The entropy of a partition of unity

A partition of unity can be thought of as a blurred out set partition: identify the fibers of the set partition with their indicator functions; the sum of these indicators is $1$. I would like to ...

**0**

votes

**1**answer

89 views

### Entropy of a refinement of a partition

We consider a probability space $(X, B, \mu)$. Let $\alpha$ and $\beta$ be countable partitions of X. We suppose $\beta$ is a refinement of $\alpha$, ie that every set in $\alpha$ is a union of sets ...

**5**

votes

**2**answers

252 views

### Shannon entropy and doubly stochastic matrices

Suppose that $A$ is a stochastic matrix. We know that if $A$ is doubly stochastic, then $H(Ap)\geq H(p)$ where $H$ is Shannon entropy and $p$ is a probability vector. Is the converse true? i.e., if $H(...

**2**

votes

**0**answers

51 views

### Entropy of flow and time-1 map

Let $\Phi=(\phi_t)_{t\in \mathbb{R}}$ be a continuous flow on a compact metric space $X$. Let $\mu$ be a $\phi_1$-invariant measure. Then it is not hard to verify tht $\int_{0}^{1} \phi_t\mu dt$ is $\...

**0**

votes

**0**answers

300 views

### Does additive Gaussian noise preserves the Shannon entropy ordering?

Suppose that $Z$ is a Gaussian random variable independent of $X$ and $Y$. Moreover suppose that $h(X) \geq h(Y)$, where $h(\cdot)$ is the differential Shannon entropy.
Does relation $h(X+Z) \geq h(Y+...

**0**

votes

**0**answers

160 views

### Is i.i.d. uniform phase distribution maximizes the Entropy?

Define
$$\Gamma(\boldsymbol{\psi})\triangleq \begin{bmatrix}e^{i\psi_1}&0&\cdots&0\\0&e^{i\psi_2}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\0&0&\cdots&e^{...

**24**

votes

**3**answers

1k views

### Is there a 'certainty' principle?

Heisenberg's uncertainty principle is a restriction on which probability distributions can describe the position and momentum of a quantum particle.
In mathematical terms it says that if $\psi\in L^2$ ...

**1**

vote

**1**answer

153 views

### Trace entropies

I'm studying relationships between trace entropy functionals and combinatorics and I'm faced with the following problem. Lets $\mathcal {D}$ be the following differential operator $1 -x\cdot \cfrac{d}{...

**1**

vote

**1**answer

118 views

### bracketing number vs covering number

Just want to double check if the lemma on page 9 of this slides is correct:
http://www.math.leidenuniv.nl/~avdvaart/talks/09hilversum.pdf
Lemma: $N(\epsilon,\cal F,||\cdot||)\leq N_{[]}(2\epsilon,\cal ...

**5**

votes

**2**answers

336 views

### Explanation for why an ideal fluid doesn't have increasing entropy?

The equations of motion for a very simple ideal fluid (specifically a calorically perfect, monatomic, ideal gas) are \begin{align*}\dot{\rho}+\nabla \cdot (\rho u)=0 \;&\text{(mass conservation)} \...

**8**

votes

**3**answers

498 views

### Discrete entropy of the integer part of a random variable

Let $X$ be a real valued random variable. Of course, the integer part $\lfloor X \rfloor$ of $X$ is a discrete random variable taking values in $\mathbb{Z}$. We can therefore define its discrete ...

**2**

votes

**1**answer

92 views

### Conditional entropy - solve example

Given a random variable $X$ that is uniformly distributed on $[-b,b]$ and $Y=g(X)$ with
$$g(x) = \begin{cases} 0, ~~~ x\in [-c,c] \\ x, ~~~ \text{else}\end{cases}$$
Now I want to compute the ...

**0**

votes

**1**answer

102 views

### Friedland metric entropy

I was asking if it is possible to extend the definition of topological Friedland entropy for $\mathbb{Z}^d$ continuos actions to measure preserving actions.
The topologica Friedland entropy is ...

**1**

vote

**0**answers

61 views

### Entropy of Markov measure using marginal distribution

Let $S$ be a finite set and let $\mathcal{X}$ be the set of all bi-infinite sequences over $S$. Let $\eta_1,\eta_2$ be two shift invariant 1-step Markov measures over $\mathcal{X}$. For a finite word $...

**1**

vote

**1**answer

137 views

### Existence of the differential entropy for infinitely divisible laws

Let $X$ be an absolutely continuous (i.e. its law is absolutely continuous with respect to the Lebesgue measure) random variable with probability density $p$. Its differential entropy is given by
$$h(...

**2**

votes

**1**answer

102 views

### Information theory for uncountably infinite-dimensional continuous random variable

I'm exploring the possibility to apply information theory on an uncountably infinite-dimensional scenario. I found the concept of generalized entropy for continuous random variables defined on finite-...

**0**

votes

**0**answers

150 views

### Is there a difference between using nats and bits to express entropy?

It seems to me like questions involving decimal vs binary representations of some number are not particularly interesting: for instance $\pi$ or $\sqrt{2}$ are conjectured to be normal in every base, ...

**2**

votes

**0**answers

97 views

### Characterization of topological entropy?

Let $V$ be a smooth Anosov vector field on a compact $n$ dimensional manifold $X$. Let $D(X)$ denote the set of distance functions $d$ on $X$ that are equivalent to fixed Riemannian distance. For each ...

**2**

votes

**1**answer

331 views

### Concavity of entropy difference

Suppose that $\mathrm{A}$ is a $n\times n$ random matrix with a given distribution. Suppose that $\mathrm{U}$ is a diagonal unitary random matrix, defined as
\begin{align*}
\begin{bmatrix}
\exp(i\...

**1**

vote

**1**answer

96 views

### Extension of the definition of entropy to $\mathbb{Z}^d$ and $\mathbb{N}^d$

I read the paper Entropie d'un groupe abélien de transformation by Conze and the part of the book Dynamical systems of Algebraic Origin by Schmidt about the entropy for $\mathbb{Z}^d$ actions. I was ...

**1**

vote

**0**answers

39 views

### Differential entropy under the change-of-variable with additive Gaussian noise

I have two Gaussian random variables $$X \sim \mathcal N(0, I), \ \ \ W \sim \mathcal N(0, \sigma\cdot I)$$ and I known a parametric change-of-variable $Y(\theta) = T(X; \theta)$.
I would like to ...

**1**

vote

**0**answers

70 views

### Covering number for the unit ball in a reproducing kernel Hilbert space

I am looking for a reference for an upper bound on the covering number for the unit ball $\{ f \in \mathcal{H}: ||f||_{\mathcal{H}} || \leq 1\} $, where $\mathcal{H}$ is a reproducing kernel Hilbert ...

**6**

votes

**1**answer

200 views

### Heat flow, decay of the Fisher information, and $\lambda$-displacement convexity

In the whole post I will work in the flat torus $\mathbb T^d=\mathbb R^d/\mathbb Z^d$ and $\rho$ will stand for any probability measure $\mathcal P(\mathbb T^d)$. This question is strongly related to ...

**9**

votes

**0**answers

93 views

### Factor map between subshifts preserving topological pressure (or measure-theoretic entropy)

Let $G$ be a countable amenable group and let $X,Y$ be subshifts with finite alphabet over $G$. Suppose that $h(X) = h(Y)$ (equal topological entropy). I am interested in continuous factor maps $\pi: ...

**2**

votes

**1**answer

121 views

### Applying the Abramov-Rokhlin skew product entropy formula to a bounded-to-one factor

Let $(X, \mathcal{B}, \mu, S)$ and $(Y, \mathcal{C}, \nu, T)$ be invertible probability-measure-preserving systems, with a measurable factor map $\pi: X \to Y$, i.e. $\pi \circ S = T \circ \pi$. ...

**1**

vote

**0**answers

55 views

### More formulas for joint entropy and for trace form entropies

Linked to some applications of entropy to combinatorics I'm looking for formulas expressing the joint entropy of two r. v. as a function of the conditional entropy . For example
For BWS extensive ...

**2**

votes

**0**answers

106 views

### Gurevich's entropy and topological entropy in a countable Markov shift

Good afternoon, I understand that Gurevich's entropy and topological entropy coincide when the countable Markov shift is topologically mixing (right?)
Does anyone know of an example or a reference ...

**1**

vote

**2**answers

101 views

### A conjecture on 'truncated joint moments' of binomial coefficients under binomial distribution

This is similar in spirit to Sum of squares of middle binomial sums or 'Truncated mean' of binomial coefficients under binomial distribution but gives some total estimates. Though the other ...

**2**

votes

**1**answer

155 views

### Sum of squares of middle binomial sums or 'Truncated mean' of binomial coefficients under binomial distribution

$\mu=1+\epsilon$ where $\epsilon>0$ holds.
1.Is there a good bound for $$T=\frac{\sum_{i=-\sqrt{\mu n\ln n}}^{\sqrt{\mu n\ln n}}\binom{n}{\frac n2 +i}^2}{2^n}?$$
This quantity can be ...

**2**

votes

**1**answer

124 views

### Tight sublinear estimates for a triple partial binomial summation

Is there tight estimates for the following logarithmic summation ($\gamma,\gamma'\in(0,1)$ and $\mu,\mu'>0$)
$$\log_2\Bigg(\sum_{t=\frac{n^{}}2-n^\gamma\sqrt{\mu\ln n}}^{\frac{n^{}}2+n^\gamma\sqrt{...

**1**

vote

**1**answer

147 views

### With only two characters allowed, is it possible to efficiently reference a 256 character alphabet in a string?

Let us use 0 and 1 for the binary parallel.
You have 256 characters you need to reference, imagining a 256 character alphabet. You can only use a string to reference them that contains 0 and 1. The ...

**0**

votes

**1**answer

95 views

### Terminology and approximation to logarithm of a sum of products of binomial coefficients

Denote $$T(m)=\sum_{1\leq n_m\leq n_{m-1}\leq\dots\leq n_2\leq n_1\leq m}\prod_{i=1}^{m}\binom{n_i}{n_{i+1}}.$$
Is there a name for this kind of summation and is there a good estimate for $\ln T(m)$ ...

**0**

votes

**2**answers

197 views

### Interpolating asymptotic expression for logarithm of middle binomial sums

Define $S(k,2n)=\sum_{i=-k}^k\binom{2n}{n+i}$ at every $k\in\{0,\dots,n\}$.
We know $$\ln(S(\gamma\mbox{ } n^\gamma,2n))\asymp(2n\ln2-\frac12\ln\pi-\frac12\ln n)$$ at $\gamma\rightarrow0$ and $$\ln(S(...

**9**

votes

**0**answers

235 views

### Entropy, magnitude, diversity of finite metric spaces in number theory

I was reading the article by Tom Leinster, (Maximizing
diversity in biology and beyond, arXiv link), and find it very interesting.
Since I was searching for entropies of finite metric spaces I found
...

**7**

votes

**3**answers

606 views

### Asymptotics of multinomial coefficients

Binomial coefficients have a well known asymptotics (https://en.wikipedia.org/wiki/Binomial_coefficient#Bounds_and_asymptotic_formulas) given by $$\binom nk\sim\binom{n}{\frac{n}{2}} e^{-d^2/(2n)} \...

**6**

votes

**2**answers

316 views

### Do mixing homeomorphisms on continua have positive entropy?

I am trying to understand relations between various measures of topological complexity. I have read that expansive homeomorphisms on continua, for example, have positive entropy. But I do not know ...

**8**

votes

**1**answer

234 views

### Connection between entropy and the set of factors of a sequence

Let $a = (a_n)_{n=0}^\infty$ be a bounded real-valued sequence. By a factor of $a$ I mean a finite block $w \in \mathbb R^l$ that appears in $a$, that is, there exists $n \geq 0$ such that $a_n a_{n+1}...

**0**

votes

**1**answer

166 views

### Bounding information of expression

Cross posted to theory exchange - https://cstheory.stackexchange.com/questions/45610/bounding-information-of-expression
Suppose $u_1,\ldots,u_n$ are uniformly iid in $\{0,1\}$.
Let $x_1,\ldots,x_n$ ...