Difference between Beta Process and Dirichlet process - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-25T07:09:11Z http://mathoverflow.net/feeds/question/9997 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/9997/difference-between-beta-process-and-dirichlet-process Difference between Beta Process and Dirichlet process Jonathan 2009-12-29T02:49:54Z 2009-12-30T06:37:08Z <p>I'm trying to understand the definition of a Beta process, as given in the paper: www.ece.duke.edu/~lcarin/Paisley_BP-FA_ICML.pdf</p> <p>The problem is that from the definition it follows that every Dirichlet process is also a beta process, which seems, ahm, wrong. Can you help me figure out what I don't understand?</p> <p>This is the definition from the paper: "Let $H_0$ be a continuous probability measure and α a positive scalar. Then for all disjoint, inﬁnitesimal partitions, $B_1 ,\ldots, B_K$, the beta process is generated as follows, $H(B_k) \sim Beta(\alpha H_0 (B_k ), \alpha(1 − H_0(B_k )))$</p> <p>with K → ∞ and $H_0(B_k)$ → 0 for $k = 1,\ldots,K$. This process is denoted $H \sim BP(\alpha H_0 )$."</p> <p>This is the definition for a Dirichlet Process (DP):</p> <p>If $X \sim DP(\alpha H_0)$ where $\alpha$ is a scalar and $H_0$ is a probability distribution, then for every finite partition $A_1,\ldots,A_K$ it follows that $(X(A_1),\ldots,X(A_K)) \sim DIR(\alpha H_0(A_1), \ldots,\alpha H_0(A_K))$.</p> <p>So let's assume that I have $X\sim DP(\alpha H_0)$. Given any partition $B_1 ,\ldots, B_K$, and any $k = 1 \ldots K$, I can define a partition $A_1 = B_k, A_2 = \Omega -B_k$ and from the DP definition it follows that </p> <p>$(X(A_1),X(A_2)) \sim DIR(\alpha H_0(A_1), \alpha H_0(A_2))$ which is equivalent to saying that $X(B_k) \sim Beta(\alpha H_0(B_k), \alpha(1-H_0(B_k)))$</p> <p>hence $X\sim BP(\alpha H_0)$. Where is my mistake?</p> http://mathoverflow.net/questions/9997/difference-between-beta-process-and-dirichlet-process/10003#10003 Answer by Gjergji Zaimi for Difference between Beta Process and Dirichlet process Gjergji Zaimi 2009-12-29T04:35:20Z 2009-12-30T06:37:08Z <p>One cannot conclude $X\sim BP(\alpha H_0)$ just by knowing the marginal distribution of each $X(B_k)$, separately. Your calculation is not wrong as the univariate marginal distribution and conditional distribution of a Dirichlet distribution are <a href="http://en.wikipedia.org/wiki/Dirichlet_process#Stick-breaking_construction" rel="nofollow">Beta distributed</a>. </p> <p>In particular, in a Dirichlet process, samples correspond to the density function $$f(\theta)=\sum \beta_i \delta_{\theta_i}$$ (Here the $\beta_i$ are constructed as $\beta_i=\beta_i'\prod_{j &lt; i}(1-\beta_j')$, and $\beta_j'\sim \text{Beta}(1,\alpha)=Y$)</p> <p>While in a Beta process, given an <b>infinitesimal</b> partition $(B_1,...,B_K)$ with $K\to \infty$ and $H(B_k)\to 0$ the samples correspond to the density function $$H(B)=\sum \pi_i\delta_{B_i}$$ where $\pi_i\sim \text{Beta}(\alpha H_0(B_i),\alpha (1-H_0(B_i)))$</p> <p>I hope you can see the difference. One more thing, the reason why the Dirichlet process is defined in terms of finitely dimensional distributions is because <a href="http://en.wikipedia.org/wiki/Kolmogorov_extension_theorem" rel="nofollow">Kolmogorov extension theorem</a> guarantees that it defines a stochastic process. Unfortunately the Beta process, does not verify the conditions of this theorem, and as a continuous time Levy process must be defined directly in the infinitesimal limit.</p> http://mathoverflow.net/questions/9997/difference-between-beta-process-and-dirichlet-process/10011#10011 Answer by Jonathan for Difference between Beta Process and Dirichlet process Jonathan 2009-12-29T08:43:14Z 2009-12-29T08:43:14Z <p>I don't understand why you say that I can't conclude $X\sim BP(\alpha H_0)$. I showed above that if $X\sim DP(\alpha H_0)$ then for every partition $B_1...B_K$, $X(B_k)\sim Beta(\alpha H_0(B_k), \alpha(1-H_0(B_k)))$. By the definition of a Beta Process it follows that $X\sim BP(\alpha H_0)$. No?</p>