Questions tagged [entropy]
The entropy tag has no usage guidance.
102
questions with no upvoted or accepted answers
12
votes
0
answers
279
views
Entropy in elimination theory, or a brief remark by Gelfand-Kapranov-Zelevinsky
In the introduction to their book "Discriminants, resultants and multidimensional determinant", the authors state a very intriguing observation concerning the coefficients of monomials appearing in $A$...
- 882
11
votes
0
answers
191
views
Factor map between subshifts preserving topological pressure (or measure-theoretic entropy)
Let $G$ be a countable amenable group and let $X,Y$ be subshifts with finite alphabet over $G$. Suppose that $h(X) = h(Y)$ (equal topological entropy). I am interested in continuous factor maps $\pi: ...
- 675
11
votes
0
answers
292
views
Entropy, magnitude, diversity of finite metric spaces in number theory
I was reading the article by Tom Leinster, (Maximizing
diversity in biology and beyond, arXiv link), and find it very interesting.
Since I was searching for entropies of finite metric spaces I found
...
user6671
11
votes
1
answer
1k
views
Bounding the entropy of a convolution
Say we have a function $f:\mathbb{Z}_2^n \to \mathbb{R}$, such that $\sum _{x\in \mathbb{Z}_2^n} f(x)^2 = 1$ (so we can think of $\{ f(x)^2\} _{x\in \mathbb{Z}_2^n}$ as a distribution). It is natural ...
user17253
10
votes
1
answer
272
views
Bibliography request: Entropy for continued fractions
Given a strictly positive real number $x$ we set $e(x)=\log(1+x)$ if $x$ is an integer and
$$e(x)=\log(1+x)+\frac{1+\lbrace x\rbrace}{1+x}\left(e(1/\lbrace x\rbrace)-\log(1+\lbrace x\rbrace)\right)$$
...
- 17.4k
8
votes
0
answers
200
views
How to categorify entropy/perplexity?
(Migrating from math.stackexchange.com per commenter suggestion)
I was reading Baez, Fritz, and Leinster's "A Characterization of Entropy in Terms of Information Loss", and wondered if, ...
- 261
6
votes
0
answers
123
views
Countable-to-one factors of measure preserving systems do not change entropy
It is known that if $\psi$ is a factor map between probability measure preserving systems $(X,\mathscr{X},\mu,T)$ and $(Y,\mathscr{Y},\nu,S)$ is countable-to-one almost everywhere, then $h(\mu,T)=h(\...
- 1,588
6
votes
1
answer
487
views
Distribution of longest run locations in a random string
Let x be a random n-bit string, and let $I ={i_1,i_2,...,i_n}$ be the starting indexes of the longest 0-runs of x, sorted in decreasing order (so $i_1$ is the starting index of the longest (~$\log n$) ...
- 61
6
votes
0
answers
378
views
Basis functions for approximation of a convex function on unit simplex
Consider the unit $D$-simplex $S^D=\left\lbrace (x_0, x_1, \ldots, x_D) \in \mathbb{R}^{D+1} \mid \sum\limits_{i=0}^{D}x_i = 1, x_i \geq 0 \right\rbrace$. I have a bounded, convex function $f:S^D\to\...
- 171
6
votes
0
answers
206
views
$q$-deformations of fundamental equation of information and entropies
Classical information theory: fundamental equation of information
In classical information theory, the information $I(A)$ of an event $A$ (any element of the $\sigma$-algebra $\mathcal F$ of a given ...
- 326
5
votes
0
answers
198
views
Entropy arguments used by Jean Bourgain
My question comes from understanding a probabilistic inequality in Bourgain's paper on Erdős simiarilty problem: Construction of sets of positive measure not containing an affine image of a given ...
- 136
5
votes
0
answers
187
views
Divergence for Bhattacharya Information matrix
The Fisher information matrix (in the scalar parameter case) can be obtained from the Kullback-Leibler divergence by
$$g(\theta) = -\frac{\partial}{\partial \theta}\frac{\partial}{\partial \theta'}D(...
- 779
5
votes
0
answers
337
views
Maximizing Renyi entropy for a certain channel
The channel under consideration is $T = A + B$, where $A$ and $B$ take on values in $\{0, 1\}$ according to a probability mass function. Let (joint) random vector $(A_1, A_2,\ldots, A_n)$ be denoted ...
5
votes
0
answers
203
views
Reference for and Properties of the $\alpha$-entropy
Let $T \colon X \to X$ be a continuous map on, say, a compact metric space $X$. Let $\mu$ be an invariant borel measure. Under suitable conditions, a result of Brin and Katok states that $\mu$-almost ...
- 475
5
votes
0
answers
813
views
Entropy conjecture for flows
The entropy conjecture for diffeomorphisms (see for example this paper) asserts that for diffeomorphisms of manifolds, the log of the spectral radius of the actions of the diffeomorphism on the ...
- 3,878
4
votes
0
answers
209
views
Looking for a counterexample for Ruelle's inequality on compact manifold
Let $M$ be a compact differentiable manifold, and $f:M\to M$ be a $C^1$- smooth diffeomorphism.
If Assume that $\mu$ be a $f$-invariant probability measure on $M$.
Then D.Ruelle proved that
$$
h_\...
- 41
4
votes
0
answers
84
views
Sharp asymptotic behavior of the metric entropy for the unit ball in Besov space
For $s>0$ and $1 \leq p,q \leq \infty$ let $B^s_{p,q}$ be the Besov space defined on $[0,1]^d$, and assume $ s > d( \frac 1 p - \frac 1 2)_+$, such that $B^s_{p,q}$ is compactly embedded in $L^2(...
4
votes
0
answers
670
views
Functional Taylor expansion for differential entropy
Consider an continuous distribution $F$ with density $f$. The (differential) Shannon entropy of $f$ is
$h(f)=-\int f(x)\log f(x) dx$.
In the literature of differential entropy estimation, ...
- 145
4
votes
0
answers
227
views
Maximazing the joint entropy given the probability of equality
Consider 2 independent random variables $X$ and $Y$ with values in $A=\{0, 1, \ldots, q-1\}$. Suppose that $P(X=Y)$ is equal to some constant $\varepsilon$.
What is the maximal entropy $H(X, Y)$?
At ...
- 41
4
votes
0
answers
90
views
What is the entropy of binomial decay?
Let's play a game. I start with $N$ indistinguishable tokens, and I wait $T$ turns. Every turn, each token has probability $p$ of disappearing. I want an analytic formula for the entropy of this ...
- 141
4
votes
0
answers
551
views
An inequality involving conditional variance and its connection to information theory
Given absolutely continuous random variables $(X, Y)$ with joint distribution $P_{XY}$, we construct $Z:=\sqrt{\gamma} Y+N_\mathsf{G}$ where $N_\mathsf{G}\sim N(0, 1)$ and is independent of $(X,Y)$ ...
- 1,109
4
votes
0
answers
2k
views
metric entropy for Lipschitz functions
Suppose $(X,d)$ is a metric space of unit diameer and let $F$ be the collection of all $1$-Lipschitz functions mapping $X$ to $[-1,1]$, equipped with the sup-norm $||\cdot||_\infty$.
I am interested ...
- 6,061
3
votes
0
answers
106
views
Differential entropy of random Gibbs measure
There is a question I have been wondering about for a while, which I have thus far not been able to resolve. The problem revolves around random Gibbs measures. I am not very well-versed in the more ...
3
votes
0
answers
131
views
Characterization of topological entropy?
Let $V$ be a smooth Anosov vector field on a compact $n$ dimensional manifold $X$. Let $D(X)$ denote the set of distance functions $d$ on $X$ that are equivalent to fixed Riemannian distance. For each ...
3
votes
0
answers
161
views
What is a natural way to extend a function from a subset of vertices to faces?
Let $n$ be a positive integer, and suppose $f$ is a probability distribution on the $2^n$ subsets of $[\![n]\!] := \{1,\ldots,n\}$. What is a "natural" way to extend $f$ to a distribution $\bar{f}$ on ...
- 6,726
3
votes
0
answers
147
views
Diffeomorphisms preserving "nice" smooth functions
Let $\mathbb{R}^2\supset D=\{(x,y)\in\mathbb{R}^2|x^2+y^2<1\}$ be the open unit disc, and $U\subset\mathbb{R}^2$ be the interior of Koch's snowflake, as constructed in Falconer's book Fractal ...
- 51
3
votes
0
answers
121
views
How large do $r$-dimensional "Kasteleyn-Temperley-Fisher" numbers grow?
I brought up a couple of combinatorial and number-theoretic items with this MO question. Now, I shall inquire on growth estimates. Recall
$$K_r(n):=\prod_{\ell_1=1}^n\cdots\prod_{\ell_r=1}^n\left(
4\...
- 41.8k
3
votes
0
answers
57
views
Is the extension of flows from the product of a system with a K system to itself relatively K?
Let $(X,\mathcal{B},\mu,T)$ be a probability measure preserving system. It is said to be a K-system if any non-trivial factor of it has positive entropy. Also we can define the notion of relatively K ...
- 395
3
votes
0
answers
151
views
On the volume entropy of negatively curved manifolds
Let $X$ be the universal cover of a closed negatively curved Riemannian manifold. Let $x_0\in X$ be a base point, $S$ be the unit sphere in $T_{x_0}X$ and $\exp:T_{x_0}X\rightarrow X$ be the ...
- 1,764
3
votes
0
answers
179
views
Is there a known generalization of the Schmidt decomposition based on a maximal set of "locally orthogonal" vectors?
I came across the following unusual generalization of the Schmidt decomposition in my work, which I describe below. I would like to know if this structure has been studied before so I can read more ...
- 846
3
votes
0
answers
179
views
A.G. Vitushkin's "Easily representable families of functions" - can it be generalized?
Background
In his monograph "Estimation of the complexity of the tabulation problem" (translated into English as "Theory of the Transmission and Processing of Information") Vitushkin studies ...
- 949
3
votes
0
answers
381
views
Does this inequality of negative relative entropy and quantum relative entropy hold?
Hello, everyone!
Question
I have a question about the relationship between general relative entropy and general quantum relative entropy: Given a unit vector $|i\rangle$ and two Hermitian matrices $...
- 607
2
votes
0
answers
102
views
Information inequality for Renyi divergences
Let $X^1 \ldots X^n$ be random variables on $\mathbb{R}^d$ with an arbitrary joint probability distribution $\mu$ on $\mathbb{R}^{n \times d}$. Let $\nu = \nu^1 \times \ldots \times \nu^n$ be a ...
2
votes
0
answers
95
views
Generalization of the min-entropy that looks at the top $n$ probabilities
The min-entropy of a random variable $X$ can often be much easier to compute than the Shannon entropy. This is because the min-entropy is simply a function of the most probable value, and sometimes, ...
- 4,525
2
votes
0
answers
122
views
List decodability of Reed-Solomon codes beyond the Johnson bound
In a paper on a proximity test for Reed-Solomon codes the authors state an "extremely optimistical" conjecture on the list decodability of Reed-Solomon codes (over prime fields $\mathbb F_q$)...
- 81
2
votes
0
answers
131
views
Entropy of a sequence
I am reading the paper Sign Changes in Hecke Eigenvalues by Matomaki and Radziwill, and in one place they mention the following,
It would be interesting to rule out the possibility of $\lambda_f(n)$ ...
- 537
2
votes
0
answers
240
views
Covering/Bracketing number of monotone functions on $\mathbb{R}$ with uniformly bounded derivatives
I am interested in the $\| \cdot \|_{\infty}$-norm bracketing number or covering number of some collection of distribution functions on $\mathbb{R}$.
Let $\mathcal{F}$ consist of all distribution ...
- 93
2
votes
0
answers
249
views
Prove or disprove a mutual information inequality
I have $n$ IID Bernoulli random variables denoted by $X_1,X_2,\ldots X_n$ with parameter $p$.
I am interested in knowing if the following inequality involving mutual information holds :
$\boxed{\max_{...
- 83
2
votes
0
answers
85
views
Continuity of the entropy of the solution of a parabolic PDE at $t=0$
Consider the following initial value problem for a parabolic PDE :
$$\begin{cases}
\textrm{div}\big(A\,\nabla u(t,x) + b(x)\, u(t,x)\big) \,=\,\partial_t u(t,x) \quad x\in\mathbb R^d\,,\ t>0 \\[4pt]...
- 311
2
votes
0
answers
118
views
A result of the covering number
Suppose $\mathcal{F} = \{f_x : x \in \mathbb{R}^d \}$ and each $f_x$ shares the same law $P$. If $\mathcal{F}$ is a class of uniformly bounded functions satisfying $L_r$-continuity, i.e. $\forall f \...
- 331
2
votes
0
answers
58
views
Maximum entropy distribution in the hyperbolic plane with given "mean" and "variance"
On the Euclidean sphere, by defining suitable analogs of the mean and variance in terms of the first circular moment, it can be shown that the von Mises distribution is the maximum entropy ...
- 265
2
votes
0
answers
84
views
Entropy of flow and time-1 map
Let $\Phi=(\phi_t)_{t\in \mathbb{R}}$ be a continuous flow on a compact metric space $X$. Let $\mu$ be a $\phi_1$-invariant measure. Then it is not hard to verify tht $\int_{0}^{1} \phi_t\mu dt$ is $\...
- 615
2
votes
0
answers
267
views
Covering number for the unit ball in a reproducing kernel Hilbert space
I am looking for a reference for an upper bound on the covering number for the unit ball $\{ f \in \mathcal{H}: ||f||_{\mathcal{H}} || \leq 1\} $, where $\mathcal{H}$ is a reproducing kernel Hilbert ...
- 121
2
votes
0
answers
176
views
Gurevich's entropy and topological entropy in a countable Markov shift
Good afternoon, I understand that Gurevich's entropy and topological entropy coincide when the countable Markov shift is topologically mixing (right?)
Does anyone know of an example or a reference ...
- 173
2
votes
0
answers
152
views
The topological complexity of polytopes
Polytopes arise naturally when modelling fundamental structures in Biology such as RNA and proteins [1,2]. Recently, it occurred to me that a complexity measure on the topology of polytopes might be ...
- 3,827
2
votes
0
answers
120
views
Example shows that entropy is not upper semi continuous
Let $(X, \beta, \mu)$ be probabilty space of compact space $X$. Let $T:X \rightarrow X$ be continuous function, and expansive. It is well known that entropy $\mu \mapsto h_{\mu}$ is upper semi ...
- 1,011
2
votes
0
answers
93
views
Can entropy of a network be written as a polynomial?
In my research, I met a problem here.
Consider a weighted graph Laplacian matrix
$$\mathcal{L}_w(\mathcal{G}) = DWD^T,$$ where $D$ is a incidence matrix and $W$ is diagonal with each diagonal ...
- 345
2
votes
0
answers
207
views
Minimising an Integrated Relative Entropy Functional
Suppose I am given
A probability distribution on $\mathbf R^d$, with density $\pi (x)$.
A family of transition kernels $\{ q^0 (x \to \cdot) \}_{x \in \mathbf R^d}$ on $\mathbf R^d$, with densities $...
- 706
2
votes
0
answers
83
views
Is Colding-Minicozzi entropy continuous w.r.t. $C^\infty$ convergenge?
For an hypersurface $\Sigma^n \subseteq \mathbb{R}^{n+1}$ the entropy introduced by Colding and Minicozzi (see their paper) is defined as
$$
\lambda(\Sigma) := \sup_{x_0 \in \mathbb{R}^{n+1} \\ t_0 \...
- 823
2
votes
0
answers
90
views
What are the moments of Kolmogorov Complexity for a Random Variable?
Given a random variable $X$ distributed under some computable distribution $P$ we have,
$$0 \le E[K(X)] - H(P) \le K(P)$$
Where $H(P)$ is the entropy of $P$. I tried using Hoeffding concentration ...
