# Questions tagged [entropy]

The tag has no usage guidance.

289 questions
Filter by
Sorted by
Tagged with
90 views

### Does the entropy of a SDE with nondegenerate noise always increase?

Let $W$ be a standard Brownian motion, and let $X$ be the solution to the one dimensional SDE $$dX_t = \sigma(t, X_t) \, dW_t$$ with initial condition $X_0 = x_0$ a.s. for some $x_0 \in \mathbb R$. We ...
25 views

### Fischer Information and Entropy, matrix case, determinant of covariance matrix going to zero

If $X \sim \mathcal{N}(\mu, \sigma^2)$, then \begin{equation} \mathcal{I}\left(\mu, \sigma^2\right)=\left(\begin{array}{cc} \frac{1}{2\mathcal{H}...
47 views

### How to prove that pseudo entropy and topological entropy are the same with only Markov inequality and continuity?

Let $(X,\rho)$ be a compact metric space and $f:X\to X$ a homeomorphism. We say $(x_1,\ldots,x_{n})\in X^n$ is a partial $n$ orbit if $f(x_i)=x_{i+1}$. Let $Sep_{\epsilon}(X,\rho_n)$ be the maximal ...
186 views

170 views

### Bibliography request: Entropy for continued fractions

Given a strictly positive real number $x$ we set $e(x)=\log(1+x)$ if $x$ is an integer and $$e(x)=\log(1+x)+\frac{1+\lbrace x\rbrace}{1+x}\left(e(1/\lbrace x\rbrace)-\log(1+\lbrace x\rbrace)\right)$$ ...
192 views

### Entropy of $f^{m(x)+n}$ of full shift

Let $(X,\mu,f)$ be a two-sided full shift system. Assume that there is $t \in \mathbb{N}$ such that for every $n \in \mathbb{N}$ and $x \in X$, we can define $T(x)=f^{n+m(x)}(x)$, where $m(x) \leq t;$...
74 views

### Generalization of the min-entropy that looks at the top $n$ probabilities

The min-entropy of a random variable $X$ can often be much easier to compute than the Shannon entropy. This is because the min-entropy is simply a function of the most probable value, and sometimes, ...
225 views

86 views

### List decodability of Reed-Solomon codes beyond the Johnson bound

In a paper on a proximity test for Reed-Solomon codes the authors state an "extremely optimistical" conjecture on the list decodability of Reed-Solomon codes (over prime fields $\mathbb F_q$)...
45 views

### Maximize entropy under Kulback-Leibler divergence

I posed this question in math.stackexchange.com, but have not received any answer. I would like to try my luck here. In this question, it is to solve \begin{align} \max_p &-\int dy\,p(y)\ln p(y) \\...
114 views

### Entropy of a sequence

I am reading the paper Sign Changes in Hecke Eigenvalues by Matomaki and Radziwill, and in one place they mention the following, It would be interesting to rule out the possibility of $\lambda_f(n)$ ...
205 views

### An inequality relating the Kullback-Leibler divergence of two discrete distributions with constant reference distribution

Suppose that $D_{KL}(p_1\parallel q)<1$ and $D_{KL}(p_2\parallel q)<1$. I'm trying to show that either $D_{KL}(p_1\parallel p_2)$ or $D_{KL}(p_2\parallel p_1)$ will have an upper bound close to ...
1 vote
194 views

234 views

### Does entropy of the random walk control the return probability

Given an infinite connected graph $G$ of bounded degree with vertex set $X$, let $P_x^n$ the time $n$ distribution of the simple random walk started at the vertex $x$ (so $P^n_x(y)$ is the probability ...
283 views

### Bipartite version of Hamming bound (two families of codewords with large Hamming distance)?

Update: In light of Fedor Petrov's answer, I added an additional requirement that all strings in $A$ and $B$ have Hamming weight exactly $n/2$, which hopefully makes the question more interesting. ...
274 views

### Fokker-Planck: equivalence between linear spectral gap and nonlinear displacement convexity?

In a smooth, bounded and convex domain $\Omega\subset \mathbb R^d$ consider the usual linear Fokker-Planck equation with Neumann (some would say Robin) boundary conditions \begin{equation} \label{FP} \...
77 views

121 views

### Convexity of the exponential of the negative Renyi entropy

I would like to try my luck here for the following question after failing to elicit an answer to it on math.stackexchange.com. For $r\ge -1$, the exponential of the negative Renyi entropy is defined ...
362 views

### 3-periodic point implies positive topological entropy

When I learn some basic ergodic theory, I encounter an interesting exercise. As we all know, 3-periodic point often means chaos. Therefore, when a continuous map has a 3-periodic point, it may have ...
109 views

### Boltzmann distribution

Let's say we have n points, on which the Boltzmann distribution $P = \{p_1,p_2,...,p_n\}$ is defined. Is it generally true that $\prod_{i=1}^n p_i < \prod_{i=1}^m q_i$ if $Q = \{q_1,...,q_m \}$ is ...
1 vote
67 views

115 views

### Zero entropy and the Koopman representation

Let $T$ be a measure preserving bijection of a probability space $(X,\nu)$. Consider the Koopman representation of $\mathbb{Z}$ on $L^2(X,\nu)$ given by $[z.f](x) = f(T^{-z}(x))$. The question is: can ...
83 views

363 views

### Difference between Shannon entropy and min-entropy

I would like to construct a sequence of discrete random variable $X_2, X_3,...,X_n,...$, where $X_n \in\{0,1,2,...,n-1\}$. Given any $\epsilon \in (0,1)$, its Shannon entropy and min-entropy should ...
199 views

### An inequality in the optimality of Bayes' theorem

$\DeclareMathOperator\Ent{Ent}\newcommand{\prior}{\mathrm{prior}}\newcommand\Data{\mathrm{Data}}$I came across this paper on the optimality of Bayes' theorem https://sinews.siam.org/Portals/Sinews2/...
1 vote
52 views

### How does a computer program recognize shocks given data of a solution to a conservation law?

Conservation laws are PDEs of the form $u_t +j_x=0.$ A discontinuous solution (for $u$ and $j$) to an equation like this can be easily found. Let's suppose that we are working with a piecewise ...
133 views

### Trying to prove an inequality (looks similar to entropy)

I'm trying to prove the following inequality (or something similar, up to a constant factor in either side of the inequality): k\cdot\sum_{i=1}^{k}x_{i}\cdot\ln\left(x_{i}\right)\geq\sum_{i=1}^{k}x_{...
1 vote
It is known that the first order error term in the Shannon entropy formula for a binomial distribution is $1/n$ (for example, see the Wikipedia page Binomial distribution), where in the limit \$n \to \... 