# Questions tagged [entropy]

The tag has no usage guidance.

234 questions
Filter by
Sorted by
Tagged with
56 views

51 views

160 views

102 views

### Information theory for uncountably infinite-dimensional continuous random variable

I'm exploring the possibility to apply information theory on an uncountably infinite-dimensional scenario. I found the concept of generalized entropy for continuous random variables defined on finite-...
150 views

### Is there a difference between using nats and bits to express entropy?

It seems to me like questions involving decimal vs binary representations of some number are not particularly interesting: for instance $\pi$ or $\sqrt{2}$ are conjectured to be normal in every base, ...
97 views

### Characterization of topological entropy?

Let $V$ be a smooth Anosov vector field on a compact $n$ dimensional manifold $X$. Let $D(X)$ denote the set of distance functions $d$ on $X$ that are equivalent to fixed Riemannian distance. For each ...
331 views

### Concavity of entropy difference

Suppose that $\mathrm{A}$ is a $n\times n$ random matrix with a given distribution. Suppose that $\mathrm{U}$ is a diagonal unitary random matrix, defined as \begin{align*} \begin{bmatrix} \exp(i\...
96 views

### Extension of the definition of entropy to $\mathbb{Z}^d$ and $\mathbb{N}^d$

I read the paper Entropie d'un groupe abélien de transformation by Conze and the part of the book Dynamical systems of Algebraic Origin by Schmidt about the entropy for $\mathbb{Z}^d$ actions. I was ...
39 views

### Differential entropy under the change-of-variable with additive Gaussian noise

I have two Gaussian random variables $$X \sim \mathcal N(0, I), \ \ \ W \sim \mathcal N(0, \sigma\cdot I)$$ and I known a parametric change-of-variable $Y(\theta) = T(X; \theta)$. I would like to ...
70 views

### Covering number for the unit ball in a reproducing kernel Hilbert space

I am looking for a reference for an upper bound on the covering number for the unit ball $\{ f \in \mathcal{H}: ||f||_{\mathcal{H}} || \leq 1\}$, where $\mathcal{H}$ is a reproducing kernel Hilbert ...
200 views

### Heat flow, decay of the Fisher information, and $\lambda$-displacement convexity

In the whole post I will work in the flat torus $\mathbb T^d=\mathbb R^d/\mathbb Z^d$ and $\rho$ will stand for any probability measure $\mathcal P(\mathbb T^d)$. This question is strongly related to ...
Let $G$ be a countable amenable group and let $X,Y$ be subshifts with finite alphabet over $G$. Suppose that $h(X) = h(Y)$ (equal topological entropy). I am interested in continuous factor maps $\pi: ... 1answer 121 views ### Applying the Abramov-Rokhlin skew product entropy formula to a bounded-to-one factor Let$(X, \mathcal{B}, \mu, S)$and$(Y, \mathcal{C}, \nu, T)$be invertible probability-measure-preserving systems, with a measurable factor map$\pi: X \to Y$, i.e.$\pi \circ S = T \circ \pi$. ... 0answers 55 views ### More formulas for joint entropy and for trace form entropies Linked to some applications of entropy to combinatorics I'm looking for formulas expressing the joint entropy of two r. v. as a function of the conditional entropy . For example For BWS extensive ... 0answers 106 views ### Gurevich's entropy and topological entropy in a countable Markov shift Good afternoon, I understand that Gurevich's entropy and topological entropy coincide when the countable Markov shift is topologically mixing (right?) Does anyone know of an example or a reference ... 2answers 101 views ### A conjecture on 'truncated joint moments' of binomial coefficients under binomial distribution This is similar in spirit to Sum of squares of middle binomial sums or 'Truncated mean' of binomial coefficients under binomial distribution but gives some total estimates. Though the other ... 1answer 155 views ### Sum of squares of middle binomial sums or 'Truncated mean' of binomial coefficients under binomial distribution$\mu=1+\epsilon$where$\epsilon>0$holds. 1.Is there a good bound for $$T=\frac{\sum_{i=-\sqrt{\mu n\ln n}}^{\sqrt{\mu n\ln n}}\binom{n}{\frac n2 +i}^2}{2^n}?$$ This quantity can be ... 1answer 124 views ### Tight sublinear estimates for a triple partial binomial summation Is there tight estimates for the following logarithmic summation ($\gamma,\gamma'\in(0,1)$and$\mu,\mu'>0$) $$\log_2\Bigg(\sum_{t=\frac{n^{}}2-n^\gamma\sqrt{\mu\ln n}}^{\frac{n^{}}2+n^\gamma\sqrt{... 1answer 147 views ### With only two characters allowed, is it possible to efficiently reference a 256 character alphabet in a string? Let us use 0 and 1 for the binary parallel. You have 256 characters you need to reference, imagining a 256 character alphabet. You can only use a string to reference them that contains 0 and 1. The ... 1answer 95 views ### Terminology and approximation to logarithm of a sum of products of binomial coefficients Denote$$T(m)=\sum_{1\leq n_m\leq n_{m-1}\leq\dots\leq n_2\leq n_1\leq m}\prod_{i=1}^{m}\binom{n_i}{n_{i+1}}.$$Is there a name for this kind of summation and is there a good estimate for \ln T(m) ... 2answers 197 views ### Interpolating asymptotic expression for logarithm of middle binomial sums Define S(k,2n)=\sum_{i=-k}^k\binom{2n}{n+i} at every k\in\{0,\dots,n\}. We know$$\ln(S(\gamma\mbox{ } n^\gamma,2n))\asymp(2n\ln2-\frac12\ln\pi-\frac12\ln n)$$at \gamma\rightarrow0 and$$\ln(S(... 0answers 235 views ### Entropy, magnitude, diversity of finite metric spaces in number theory I was reading the article by Tom Leinster, (Maximizing diversity in biology and beyond, arXiv link), and find it very interesting. Since I was searching for entropies of finite metric spaces I found ... 3answers 606 views ### Asymptotics of multinomial coefficients Binomial coefficients have a well known asymptotics (https://en.wikipedia.org/wiki/Binomial_coefficient#Bounds_and_asymptotic_formulas) given by$$\binom nk\sim\binom{n}{\frac{n}{2}} e^{-d^2/(2n)} \... 2answers 316 views ### Do mixing homeomorphisms on continua have positive entropy? I am trying to understand relations between various measures of topological complexity. I have read that expansive homeomorphisms on continua, for example, have positive entropy. But I do not know ... 1answer 234 views ### Connection between entropy and the set of factors of a sequence Let$a = (a_n)_{n=0}^\infty$be a bounded real-valued sequence. By a factor of$a$I mean a finite block$w \in \mathbb R^l$that appears in$a$, that is, there exists$n \geq 0$such that$a_n a_{n+1}...
Cross posted to theory exchange - https://cstheory.stackexchange.com/questions/45610/bounding-information-of-expression Suppose $u_1,\ldots,u_n$ are uniformly iid in $\{0,1\}$. Let $x_1,\ldots,x_n$ ...