# Tagged Questions

The tag has no usage guidance.

1k views

### System with invariant measure, but no ergodic measure.

Question Examples of continuous transformations $T: X \to X$ such that the family of invariant probability measures $M(T)$ is NOT empty but there is no ergodic measure ($E(T) = \emptyset$). Notice ...
810 views

### Supremum amongst Kolmogorov-Sinai entropies: ergodic or just invariant measures.

Cases where $sup_{\mu \in E(T)} h_\mu(T) \neq \sup_{\mu \in M(T)} h_\mu(T)$. Background For a topological space $X$, let $T: X \to X$ be a continuous application. Then, call the set of $T$-...
417 views

### closed form expression for Rényi entropy for multivariate Gaussian distributions

Is there any closed form expression for Rényi entropy of a set variables with multivariate Gaussian distribution?
330 views

### Robust entropy-like measure for analyzing uncertainity

I'm looking for a measure to analysis the uncertainty observed in a set of variables (with multivariate Gaussian distribution). So, I've tried conventional Shanon entropy (differential entropy) which ...
8k views

### How to compute KL-divergence when PMF contains 0s?

From the Wikipedia page on Kullback-Leibler divergence, the way to compute this metric is to utilize the following formula: The way I understand this is to compute the PMFs of two given sample sets ...
248 views

### A geometric/topology notion of Typical Sequences? Power of typical sequences in multiuser channels?

The idea of Typical sequences(http://en.wikipedia.org/wiki/Typical_set) is a crucial concept in Shannon's proof of the Noisy channel coding theorem. Unfortunately the notion is not sufficient to ...
2k views

### Entropy of the Ising model

Consider the standard Ising model on $[0,N]^2$ for $N$ large. By that I mean the square-lattice Ising model without external field, inside an $N$-by-$N$ square. What is its entropy for $N$ large? It ...
425 views

### Infinitely Divisible Distributions and Maximal Entropy

The normal distribution on $\mathbb{R}$, the exponential distribution on $\mathbb{R}_{\geq 0}$, and the geometric distribution on $\mathbb{N}$ are examples of distributions that are both infinitely ...
1k views

93 views

### Median entropy to observe evolution of system?

Hello, I am studying a dynamical system that takes as an initial condition a list. I want to analyze the evolution of Shannon's entropy in this system. I know the maximum entropy (50) and the minimum ...
7k views

### How to ensure the non-negativity of Kullback-Leibler Divergence KLD Metric (Relative Entropy)?

Hello, I’m having some problems in ensuring the non-negativity of KLD! I know that KLD is always positive and I went over the proof. However, it doesn’t seem to work for me. In some cases I’m ...
1k views

### Optimization of relative entropy

Wondering if my following question is an application of information theory: Lets say we have a factory and ship boxes of stuff outside. If a competitor stands outside my factory, observes the stream ...