Questions tagged [entropy]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
56 views

A result of the covering number

Suppose $\mathcal{F} = \{f_x : x \in \mathbb{R}^d \}$ and each $f_x$ shares the same law $P$. If $\mathcal{F}$ is a class of uniformly bounded functions satisfying $L_r$-continuity, i.e. $\forall f \...
4
votes
1answer
250 views

Are these topological sequence entropy definition equivalent?

I am working on Möbius disjointness for models of topological dynamic systems. In that purpose, I try to understand the notion of topological entropy. We know, for a t.d.s $(X,T)$ that it is defined ...
1
vote
1answer
50 views

Entropy condition for quasi-linear evolution equations

Let's consider the problem: $$ \partial_t u + \partial_x(f(u)) = 0, (x,t)\in \mathbb R \times \mathbb R^+.\\ u|_{t=0}=u_0. $$ I have seen three formulations for the entropy condition of this equation. ...
1
vote
0answers
55 views

Discrete Orlicz space estimate

We consider the discrete LlogL space of sequences $x=(x_i)$ such that $$\Vert x\Vert_{LL}:=\sum_i \vert x_i \log(x_i)\vert <\infty.$$ Let $x=(x_i)$ and $y=(y_i)$ two sequences in the above LlogL ...
2
votes
0answers
26 views

Maximum entropy distribution in the hyperbolic plane with given “mean” and “variance”

On the Euclidean sphere, by defining suitable analogs of the mean and variance in terms of the first circular moment, it can be shown that the von Mises distribution is the maximum entropy ...
2
votes
1answer
125 views

Is total variation distance of normalized sum of random variables to Gaussian monotonic decreasing?

Let $X_1, X_2, \ldots$ be independent and identically distributed random variables with mean $0$ and variance $1$ and let $S_n = (X_1 + \cdots + X_n)/\sqrt{n}$ to be their normalized sum. Define $D_n$ ...
0
votes
1answer
110 views

Existence of sequence of distributions

This question concerns distributions $\mu$ over the naturals $\mathbb{N}=\{1,2,\ldots\}$. For $q\ge1$, let us define the $q$th moment of entropy: $$ H_q(\mu)=\sum_{i=1}^\infty \mu(i)|\log\mu(i)|^q, $$ ...
1
vote
0answers
28 views

Marginal distribution of $I$-projection

I am reading this paper by Csiszar. Given a probability measure $R$ and a convex subset $\mathcal{E}$ of probability distributions, it defines ‘I-projection of R on $\mathcal{E}$’ (provided there ...
8
votes
1answer
299 views

Relative Entropy and p-norm

I asked this question on StackExchange but could not get any answer, therefore, I am posting it here. I am currently reading the book "A Dynamical Approach to Random Matrix Theory". The ...
0
votes
1answer
170 views

Integrability of $\int \log(f(x)) f(x) dx$ for a probability density function $f$

I am looking for weak conditions when a probability density function $f$ on $\mathbb{R}^d$ has a finite integral $$ \int_{\mathbb{R}^d} \log(f(x)) f(x) dx. $$ Any references would be appreciated.
0
votes
0answers
21 views

Proof of property for Fiedland entropy

I am working with Friedland entropy and there is a proof I cannot figure out how to do. Friedland entropy is defined for $\mathbb{Z}^k$ continuos actions $\mathcal{T}$ on a topological metric space $X$...
5
votes
3answers
344 views

Is there a quantum analog of Kolmogorov Complexity?

Kolmogorov Complexity (interpreted in terms of shortest program computing a string) and Shannon Entropy are quite similar. Since there is a quantum entropy is it reasonable to ask if there is quantum ...
8
votes
1answer
268 views

The entropy of a partition of unity

A partition of unity can be thought of as a blurred out set partition: identify the fibers of the set partition with their indicator functions; the sum of these indicators is $1$. I would like to ...
0
votes
1answer
89 views

Entropy of a refinement of a partition

We consider a probability space $(X, B, \mu)$. Let $\alpha$ and $\beta$ be countable partitions of X. We suppose $\beta$ is a refinement of $\alpha$, ie that every set in $\alpha$ is a union of sets ...
5
votes
2answers
252 views

Shannon entropy and doubly stochastic matrices

Suppose that $A$ is a stochastic matrix. We know that if $A$ is doubly stochastic, then $H(Ap)\geq H(p)$ where $H$ is Shannon entropy and $p$ is a probability vector. Is the converse true? i.e., if $H(...
2
votes
0answers
51 views

Entropy of flow and time-1 map

Let $\Phi=(\phi_t)_{t\in \mathbb{R}}$ be a continuous flow on a compact metric space $X$. Let $\mu$ be a $\phi_1$-invariant measure. Then it is not hard to verify tht $\int_{0}^{1} \phi_t\mu dt$ is $\...
0
votes
0answers
300 views

Does additive Gaussian noise preserves the Shannon entropy ordering?

Suppose that $Z$ is a Gaussian random variable independent of $X$ and $Y$. Moreover suppose that $h(X) \geq h(Y)$, where $h(\cdot)$ is the differential Shannon entropy. Does relation $h(X+Z) \geq h(Y+...
0
votes
0answers
160 views

Is i.i.d. uniform phase distribution maximizes the Entropy?

Define $$\Gamma(\boldsymbol{\psi})\triangleq \begin{bmatrix}e^{i\psi_1}&0&\cdots&0\\0&e^{i\psi_2}&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\0&0&\cdots&e^{...
24
votes
3answers
1k views

Is there a 'certainty' principle?

Heisenberg's uncertainty principle is a restriction on which probability distributions can describe the position and momentum of a quantum particle. In mathematical terms it says that if $\psi\in L^2$ ...
1
vote
1answer
153 views

Trace entropies

I'm studying relationships between trace entropy functionals and combinatorics and I'm faced with the following problem. Lets $\mathcal {D}$ be the following differential operator $1 -x\cdot \cfrac{d}{...
1
vote
1answer
118 views

bracketing number vs covering number

Just want to double check if the lemma on page 9 of this slides is correct: http://www.math.leidenuniv.nl/~avdvaart/talks/09hilversum.pdf Lemma: $N(\epsilon,\cal F,||\cdot||)\leq N_{[]}(2\epsilon,\cal ...
5
votes
2answers
336 views

Explanation for why an ideal fluid doesn't have increasing entropy?

The equations of motion for a very simple ideal fluid (specifically a calorically perfect, monatomic, ideal gas) are \begin{align*}\dot{\rho}+\nabla \cdot (\rho u)=0 \;&\text{(mass conservation)} \...
8
votes
3answers
498 views

Discrete entropy of the integer part of a random variable

Let $X$ be a real valued random variable. Of course, the integer part $\lfloor X \rfloor$ of $X$ is a discrete random variable taking values in $\mathbb{Z}$. We can therefore define its discrete ...
2
votes
1answer
92 views

Conditional entropy - solve example

Given a random variable $X$ that is uniformly distributed on $[-b,b]$ and $Y=g(X)$ with $$g(x) = \begin{cases} 0, ~~~ x\in [-c,c] \\ x, ~~~ \text{else}\end{cases}$$ Now I want to compute the ...
0
votes
1answer
102 views

Friedland metric entropy

I was asking if it is possible to extend the definition of topological Friedland entropy for $\mathbb{Z}^d$ continuos actions to measure preserving actions. The topologica Friedland entropy is ...
1
vote
0answers
61 views

Entropy of Markov measure using marginal distribution

Let $S$ be a finite set and let $\mathcal{X}$ be the set of all bi-infinite sequences over $S$. Let $\eta_1,\eta_2$ be two shift invariant 1-step Markov measures over $\mathcal{X}$. For a finite word $...
1
vote
1answer
137 views

Existence of the differential entropy for infinitely divisible laws

Let $X$ be an absolutely continuous (i.e. its law is absolutely continuous with respect to the Lebesgue measure) random variable with probability density $p$. Its differential entropy is given by $$h(...
2
votes
1answer
102 views

Information theory for uncountably infinite-dimensional continuous random variable

I'm exploring the possibility to apply information theory on an uncountably infinite-dimensional scenario. I found the concept of generalized entropy for continuous random variables defined on finite-...
0
votes
0answers
150 views

Is there a difference between using nats and bits to express entropy?

It seems to me like questions involving decimal vs binary representations of some number are not particularly interesting: for instance $\pi$ or $\sqrt{2}$ are conjectured to be normal in every base, ...
2
votes
0answers
97 views

Characterization of topological entropy?

Let $V$ be a smooth Anosov vector field on a compact $n$ dimensional manifold $X$. Let $D(X)$ denote the set of distance functions $d$ on $X$ that are equivalent to fixed Riemannian distance. For each ...
2
votes
1answer
331 views

Concavity of entropy difference

Suppose that $\mathrm{A}$ is a $n\times n$ random matrix with a given distribution. Suppose that $\mathrm{U}$ is a diagonal unitary random matrix, defined as \begin{align*} \begin{bmatrix} \exp(i\...
1
vote
1answer
96 views

Extension of the definition of entropy to $\mathbb{Z}^d$ and $\mathbb{N}^d$

I read the paper Entropie d'un groupe abélien de transformation by Conze and the part of the book Dynamical systems of Algebraic Origin by Schmidt about the entropy for $\mathbb{Z}^d$ actions. I was ...
1
vote
0answers
39 views

Differential entropy under the change-of-variable with additive Gaussian noise

I have two Gaussian random variables $$X \sim \mathcal N(0, I), \ \ \ W \sim \mathcal N(0, \sigma\cdot I)$$ and I known a parametric change-of-variable $Y(\theta) = T(X; \theta)$. I would like to ...
1
vote
0answers
70 views

Covering number for the unit ball in a reproducing kernel Hilbert space

I am looking for a reference for an upper bound on the covering number for the unit ball $\{ f \in \mathcal{H}: ||f||_{\mathcal{H}} || \leq 1\} $, where $\mathcal{H}$ is a reproducing kernel Hilbert ...
6
votes
1answer
200 views

Heat flow, decay of the Fisher information, and $\lambda$-displacement convexity

In the whole post I will work in the flat torus $\mathbb T^d=\mathbb R^d/\mathbb Z^d$ and $\rho$ will stand for any probability measure $\mathcal P(\mathbb T^d)$. This question is strongly related to ...
9
votes
0answers
93 views

Factor map between subshifts preserving topological pressure (or measure-theoretic entropy)

Let $G$ be a countable amenable group and let $X,Y$ be subshifts with finite alphabet over $G$. Suppose that $h(X) = h(Y)$ (equal topological entropy). I am interested in continuous factor maps $\pi: ...
2
votes
1answer
121 views

Applying the Abramov-Rokhlin skew product entropy formula to a bounded-to-one factor

Let $(X, \mathcal{B}, \mu, S)$ and $(Y, \mathcal{C}, \nu, T)$ be invertible probability-measure-preserving systems, with a measurable factor map $\pi: X \to Y$, i.e. $\pi \circ S = T \circ \pi$. ...
1
vote
0answers
55 views

More formulas for joint entropy and for trace form entropies

Linked to some applications of entropy to combinatorics I'm looking for formulas expressing the joint entropy of two r. v. as a function of the conditional entropy . For example For BWS extensive ...
2
votes
0answers
106 views

Gurevich's entropy and topological entropy in a countable Markov shift

Good afternoon, I understand that Gurevich's entropy and topological entropy coincide when the countable Markov shift is topologically mixing (right?) Does anyone know of an example or a reference ...
1
vote
2answers
101 views

A conjecture on 'truncated joint moments' of binomial coefficients under binomial distribution

This is similar in spirit to Sum of squares of middle binomial sums or 'Truncated mean' of binomial coefficients under binomial distribution but gives some total estimates. Though the other ...
2
votes
1answer
155 views

Sum of squares of middle binomial sums or 'Truncated mean' of binomial coefficients under binomial distribution

$\mu=1+\epsilon$ where $\epsilon>0$ holds. 1.Is there a good bound for $$T=\frac{\sum_{i=-\sqrt{\mu n\ln n}}^{\sqrt{\mu n\ln n}}\binom{n}{\frac n2 +i}^2}{2^n}?$$ This quantity can be ...
2
votes
1answer
124 views

Tight sublinear estimates for a triple partial binomial summation

Is there tight estimates for the following logarithmic summation ($\gamma,\gamma'\in(0,1)$ and $\mu,\mu'>0$) $$\log_2\Bigg(\sum_{t=\frac{n^{}}2-n^\gamma\sqrt{\mu\ln n}}^{\frac{n^{}}2+n^\gamma\sqrt{...
1
vote
1answer
147 views

With only two characters allowed, is it possible to efficiently reference a 256 character alphabet in a string?

Let us use 0 and 1 for the binary parallel. You have 256 characters you need to reference, imagining a 256 character alphabet. You can only use a string to reference them that contains 0 and 1. The ...
0
votes
1answer
95 views

Terminology and approximation to logarithm of a sum of products of binomial coefficients

Denote $$T(m)=\sum_{1\leq n_m\leq n_{m-1}\leq\dots\leq n_2\leq n_1\leq m}\prod_{i=1}^{m}\binom{n_i}{n_{i+1}}.$$ Is there a name for this kind of summation and is there a good estimate for $\ln T(m)$ ...
0
votes
2answers
197 views

Interpolating asymptotic expression for logarithm of middle binomial sums

Define $S(k,2n)=\sum_{i=-k}^k\binom{2n}{n+i}$ at every $k\in\{0,\dots,n\}$. We know $$\ln(S(\gamma\mbox{ } n^\gamma,2n))\asymp(2n\ln2-\frac12\ln\pi-\frac12\ln n)$$ at $\gamma\rightarrow0$ and $$\ln(S(...
9
votes
0answers
235 views

Entropy, magnitude, diversity of finite metric spaces in number theory

I was reading the article by Tom Leinster, (Maximizing diversity in biology and beyond, arXiv link), and find it very interesting. Since I was searching for entropies of finite metric spaces I found ...
7
votes
3answers
606 views

Asymptotics of multinomial coefficients

Binomial coefficients have a well known asymptotics (https://en.wikipedia.org/wiki/Binomial_coefficient#Bounds_and_asymptotic_formulas) given by $$\binom nk\sim\binom{n}{\frac{n}{2}} e^{-d^2/(2n)} \...
6
votes
2answers
316 views

Do mixing homeomorphisms on continua have positive entropy?

I am trying to understand relations between various measures of topological complexity. I have read that expansive homeomorphisms on continua, for example, have positive entropy. But I do not know ...
8
votes
1answer
234 views

Connection between entropy and the set of factors of a sequence

Let $a = (a_n)_{n=0}^\infty$ be a bounded real-valued sequence. By a factor of $a$ I mean a finite block $w \in \mathbb R^l$ that appears in $a$, that is, there exists $n \geq 0$ such that $a_n a_{n+1}...
0
votes
1answer
166 views

Bounding information of expression

Cross posted to theory exchange - https://cstheory.stackexchange.com/questions/45610/bounding-information-of-expression Suppose $u_1,\ldots,u_n$ are uniformly iid in $\{0,1\}$. Let $x_1,\ldots,x_n$ ...

1
2 3 4 5