7
votes
Accepted
Relative entropy equality for a sequence of Bernoulli random variables
This equality does not hold in general. E.g., suppose that $n=2$,
$$\big(P_p(X_1=i,X_2=j)\colon\; i=0,1,\,j=0,1\big)=
\frac1{30}\,\begin{pmatrix}
4 & 9 \\
9 & 8 \\
\end{pmatrix},$$
and
$$\...
- 118k
4
votes
Accepted
Maximum entropy probability distribution with fixed interval and variance?
Denote the support by $[a,b]$ and the variance by $\sigma^2$. Repeating the variational calculus argument that shows the normal distribution maximizes the differential entropy (see here for instance), ...
- 2,817
3
votes
Accepted
Is the Boltzmann entropy lower semi-continuous in the weak topology induced by $C_b (\mathbb R^d)$?
$\newcommand\PP{\mathcal P}\newcommand\H{\mathcal H}\newcommand\R{\mathbb R}\newcommand\si{\sigma}\newcommand\ga{\gamma}$1. The answer to Question 1 is no. Indeed, let $d=1$ and
$$\rho_n:=c_n p_n,$$
...
- 118k
3
votes
Does every proximal dynamical system have zero topological entropy?
Maybe there is an easier example, but here is an example of a proximal system with positive entropy. The dynamical system is a so-called subshift $(X, \sigma)$, where $X$ is a closed shift-invariant ...
- 2,423
3
votes
Accepted
Morse-Hedlund\Coven-Hedlund theorem for non-Abelian groups
Your motivation is about aperiodicity implying high complexity, but you ask whether aperiodicity implies an upper bound on complexity, which is a little strange. Probably there is a typo, and $O(g(r))$...
- 6,337
3
votes
Accepted
Bound on an integral representing a difference of two relative entropies
Yes, the inequality is true. Indeed, the inequality in question can be rewritten (or, if you prefer, generalized) as
\begin{equation}
Eg(Y)\ge\ln(1-a^3), \tag{10}\label{10}
\end{equation}
where
$$...
- 118k
2
votes
Accepted
Does the entropy of a SDE with nondegenerate noise always increase?
$\newcommand{\si}{\sigma}\newcommand{\R}{\mathbb R}\newcommand{\pa}{\partial}$The answer is no. The idea is to get a diffusion version of my two-state Markov chain example.
Indeed, for $t\in(0,\infty)$...
- 118k
2
votes
Accepted
Reference request: log Sobolev inequality for uniform measure (uniform distribution over discrete set)
One can use the results by Diaconis & Saloff-Coste "Logarithmic Sobolev inequalities for finite Markov chains" in Section 4.2, where the proof the result for the symmetric random walk on ...
- 1,081
2
votes
Inequalities involving entropy: quantum discord and mutual information
This is only a partial answer for now. I will sketch the answer of Question 1 in the case when all Hilbert spaces involved are finite-dimensional for simplicity.
The key point how to use concavity is ...
- 1,027
2
votes
Accepted
Is the Boltzmann entropy continuous in the supremum norm?
The answer is no. For instance, let $d=1$, $\rho(x):=e^{-x}\,1(x>0)$, $$\rho_n(x):=c_n\big(e^{-x}\,1(0<x\le n)+p_n\,1(n<x\le2n)\big),$$
where $c_n:=1/(1-e^{-n}+np_n)$, $p_n\in(0,\infty)$ for ...
- 118k
2
votes
Accepted
Defining a measure of uniformity for measurable subsets of $[0,1]^2$ w.r.t dimension $\alpha\in[0,2]$
Here is another possible approach, perhaps closer to what the OP had in mind.
Let $S:=[0,1]^2$ be the unit square. "Partition" $S$ naturally into four congruent squares $S_{1,j}$ (with side ...
- 118k
2
votes
Accepted
Can we lower bound this entropy by $\int_{\mathbb R^d} \rho^k (x) \, \mathrm d x$ and $\int_{\mathbb R^d} |x|^2\rho (x) \, \mathrm d x$?
$\newcommand\H{\mathcal H}\newcommand\R{\mathbb R}$Note that $U(s)\ge-1/e+1-s$ for all real $s\ge0$ (assuming that your $\log$ is $\ln$). So, for any probability density function $\rho$,
$$\H(\rho)\ge\...
- 118k
2
votes
Is the Boltzmann entropy lower semi-continuous in the weak topology induced by $C_b (\mathbb R^d)$?
$\newcommand\PP{\mathcal P}\newcommand\H{\mathcal H}\newcommand\R{\mathbb R}\newcommand\si{\sigma}\newcommand\ga{\gamma}$The answer to Question 1 becomes yes if, as suggested by the OP in a personal ...
- 118k
1
vote
Accepted
Let $\mu : [0, T] \to \mathcal P_2^a (\mathbb R^d), t \mapsto \mu_t$ be absolutely continuous. Is $t \mapsto \mathcal H (\mu_t)$ continuous?
$\newcommand{\R}{\mathbb R}$The answer is NO. I will provide below a counterexample in dimension $d=1$.
Preliminaries:
Let's agree that the entropy is
$$
H(\rho)=\int_{\mathbb R}\rho(x)\log\rho(x) dx
...
- 5,163
1
vote
Maximal entropy distribution on three variables knowing its marginals on any two
This is only about $|X|=|Y|=|Z|=2$ case: let $X=Y=Z=\{0,1\}$. And $f$ be a "seed" joint distribution on $X\times Y\times Z$. Then if another distribution has same pairwise marginals, there ...
- 291
1
vote
Defining a measure of uniformity for measurable subsets of $[0,1]^2$ w.r.t dimension $\alpha\in[0,2]$
Responding to the latest comment by the OP: "How would you suggest measuring the uniformity of measurable subsets of the unit square?" :
I think the idea of uniformity has hardly anything ...
- 118k
1
vote
Accepted
Example of finite closed cover with entropy strictly greater than topological entropy
You can find an example of a cascade which has topological entropy equal to zero, and which has a finite closed cover having entropy equal to $\log 2$ in the article "The Product Theorem for ...
- 118
Only top scored, non community-wiki answers of a minimum length are eligible