Maximum entropy priors in infinite dimensional spaces - MathOverflow most recent 30 from http://mathoverflow.net 2013-06-19T20:52:43Z http://mathoverflow.net/feeds/question/109671 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/109671/maximum-entropy-priors-in-infinite-dimensional-spaces Maximum entropy priors in infinite dimensional spaces Nick Alger 2012-10-15T02:53:12Z 2012-10-16T06:37:48Z <p>Is there an extension of <a href="http://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution" rel="nofollow">maximum entropy probability distributions</a> for function spaces?</p> <p>For $\mathbb{R}^n$ and discrete spaces, there is much literature about this problem under names such as "non-informative priors", "maximum entropy distributions", "Jeffrey's priors", and the like.</p> <p>There is an <a href="http://www.mdpi.com/1099-4300/11/2/222/pdf" rel="nofollow">extension</a> to locally compact topological groups, where the Haar measure $U$ takes the place of the Lebesgue measure, and one looks for measures $P$ minimizing the information divergence, $$D(P||U):= \begin{cases} \int log \frac{dP}{dU} dP, &amp; \text{ if } P\ll U; \\ \infty, &amp; \text{else.}\end{cases}$$</p> <p>However, I've found little about this in the infinite dimensional setting. Can the concept of maximum entropy priors be generalized to (some class of) function spaces, or is the idea of entropy fundamentally incompatible with spaces that are not locally compact?</p> <p>Notes,</p> <ul> <li>This is a repost from <a href="http://stats.stackexchange.com/questions/39220/maximum-entropy-priors-in-infinite-dimensional-spaces" rel="nofollow">this statistics stackexchange thread</a> which got no answers.</li> <li>There is an <a href="http://www.itp.uni-hannover.de/~furrer/MasterThesisV2.pdf" rel="nofollow">infinite dimensional generalization</a> of the "min- entropy" $H_\infty$, though this is a different concept from the "minimum entropy" $\text{min } H$.</li> </ul> http://mathoverflow.net/questions/109671/maximum-entropy-priors-in-infinite-dimensional-spaces/109674#109674 Answer by R Hahn for Maximum entropy priors in infinite dimensional spaces R Hahn 2012-10-15T03:23:07Z 2012-10-15T03:23:07Z <p>Cover and Thomas's Elements of Information Theory has a chapter on maximum entropy stochastic processes. The relevant quantity in that case is the entropy <em>rate</em>. See section 12.5, for example, which is visible in Google books.</p> http://mathoverflow.net/questions/109671/maximum-entropy-priors-in-infinite-dimensional-spaces/109678#109678 Answer by an12 for Maximum entropy priors in infinite dimensional spaces an12 2012-10-15T04:17:02Z 2012-10-16T06:37:48Z <p>There is work on infinite-dimensional exponential families of measures which might be what you are looking for.</p> <p>There are these possible references:</p> <ul> <li><a href="http://projecteuclid.org/euclid.aos/1179935069" rel="nofollow">Scricciolo (2006). Convergence rates for Bayesian density estimation of infinite-dimensional exponential families</a></li> <li><a href="http://projecteuclid.org/euclid.ba/1339878890" rel="nofollow">Rivoirard and Rousseau (2012). Posterior concentration rates for infinite dimensional exponential families</a></li> <li><a href="http://www.springerlink.com/index/N2271QH5P77764N8.pdf" rel="nofollow">Deuschel (1987). Infinite-dimensional diffusion processes as Gibbs measures on $C \left[ 0, 1 \right]^{Z^d}$</a></li> </ul> <p>The first paper provides such distributions on Sobolev spaces. Also, the references in that paper are helpful about using infinite-dimensional entropy optimizing-based measures in order to construct priors for use in Bayesian non-parametric statistics.</p> <p><strong>Edit</strong>: I added an additional reference that is also relevant to the topic.</p>