The entropy tag has no wiki summary.

**1**

vote

**1**answer

359 views

### existence of finitely additive measures with zero entropy

Let $X$ be a countable set and $\mathcal M(X)$ be the set of finitely additive probability measures on $X$. If $\mu\in\mathcal M(X)$, I define the entropy of $\mu$ to be
$$
...

**2**

votes

**1**answer

207 views

### Compact group extension of a zero entropy system.

Suppose $T: X \to X$ is a continuous map
and $\mu$ a $T$-ergodic probability measure over the
Borel sets of $X$.
Now, suppose $K \subset \mathrm{Hom}(X)$ is a compact group
of measure-preserving ...

**10**

votes

**1**answer

1k views

### System with invariant measure, but no ergodic measure.

Question
Examples of continuous transformations $T: X \to X$ such that the family of invariant probability measures $M(T)$ is NOT empty but there is no ergodic measure ($E(T) = \emptyset$).
Notice ...

**10**

votes

**3**answers

710 views

### Supremum amongst Kolmogorov-Sinai entropies: ergodic or just invariant measures.

Cases where
$sup_{\mu \in E(T)} h_\mu(T)
\neq
\sup_{\mu \in M(T)} h_\mu(T)$.
Background
For a topological space $X$,
let $T: X \to X$ be a continuous application.
Then, call the set of ...

**0**

votes

**1**answer

322 views

### closed form expression for Rényi entropy for multivariate Gaussian distributions

Is there any closed form expression for Rényi entropy of a set variables with multivariate Gaussian distribution?

**0**

votes

**1**answer

288 views

### Robust entropy-like measure for analyzing uncertainity

I'm looking for a measure to analysis the uncertainty observed in a set of variables (with multivariate Gaussian distribution). So, I've tried conventional Shanon entropy (differential entropy) which ...

**0**

votes

**0**answers

621 views

### Calculate entropy for a set of data

Hi;
I am not really a maths person and I have a question regarding shannon entropy. I have different datasets which only consists of three letter(I,N,M) such as :
dataset1: {I,I,N,M,I,I}
dataset2: ...

**1**

vote

**1**answer

3k views

### How to compute KL-divergence when PMF contains 0s?

From the Wikipedia page on Kullback-Leibler divergence, the way to compute this metric is to utilize the following formula:
The way I understand this is to compute the PMFs of two given sample sets ...

**6**

votes

**1**answer

235 views

### A geometric/topology notion of Typical Sequences? Power of typical sequences in multiuser channels?

The idea of Typical sequences(http://en.wikipedia.org/wiki/Typical_set) is a crucial concept in Shannon's proof of the Noisy channel coding theorem. Unfortunately the notion is not sufficient to ...

**7**

votes

**1**answer

1k views

### Entropy of the Ising model

Consider the standard Ising model on $[0,N]^2$ for $N$ large. By that I mean the square-lattice Ising model without external field, inside an $N$-by-$N$ square. What is its entropy for $N$ large? It ...

**3**

votes

**1**answer

357 views

### Infinitely Divisible Distributions and Maximal Entropy

The normal distribution on $\mathbb{R}$, the exponential distribution on $\mathbb{R}_{\geq 0}$, and the geometric distribution on $\mathbb{N}$ are examples of distributions that are both infinitely ...

**6**

votes

**1**answer

663 views

### Entropy of first return map and suspension flows

There are some well know formulas of Abramov about derived systems.
Firstly let $(X,\mu,f)$ be a probability preserving system and $A\subset X$ is measurable such that $\bigcup_{n\ge0}f^nA=X$. Let ...

**5**

votes

**2**answers

264 views

### Entropy of nested compact invariant sets

Let $f$ be a homeomorphism on a compact metric space $X$. $K_1\supset K_2\supset\cdots \supset K$ are compact subsets of $X$ such that $f(K_n)=K_n$ and
$K=\bigcap K_n$. If $h(f, K_1)<\infty$, do we ...

**1**

vote

**0**answers

83 views

### Median entropy to observe evolution of system?

Hello,
I am studying a dynamical system that takes as an initial condition a list. I want to analyze the evolution of Shannon's entropy in this system. I know the maximum entropy (50) and the minimum ...

**0**

votes

**2**answers

3k views

### How to ensure the non-negativity of Kullback-Leibler Divergence KLD Metric (Relative Entropy)?

Hello,
I’m having some problems in ensuring the non-negativity of KLD!
I know that KLD is always positive and I went over the proof. However, it doesn’t seem to work for me. In some cases I’m ...

**3**

votes

**2**answers

846 views

### Optimization of relative entropy

Wondering if my following question is an application of information theory:
Lets say we have a factory and ship boxes of stuff outside. If a competitor stands outside my factory, observes the stream ...

**3**

votes

**0**answers

573 views

### Entropy conjecture for flows

The entropy conjecture for diffeomorphisms (see for example this paper) asserts that for diffeomorphisms of manifolds, the log of the spectral radius of the actions of the diffeomorphism on the ...

**1**

vote

**3**answers

2k views

### Entropy of Random Signal

How would you proof that for all equal variance (Continuous) Random Signals, a Gaussian one would have the largest Entropy. Or in other words, Given a Variance Gaussian PDF maximizes the Entropy.
...