Difference between Beta Process and Dirichlet process - MathOverflow

Difference between Beta Process and Dirichlet process

2009-12-29T02:49:54Z

I'm trying to understand the definition of a Beta process, as given in the paper: www.ece.duke.edu/~lcarin/Paisley_BP-FA_ICML.pdf

The problem is that from the definition it follows that every Dirichlet process is also a beta process, which seems, ahm, wrong. Can you help me figure out what I don't understand?

This is the definition from the paper: "Let $H_0$ be a continuous probability measure and α a positive scalar. Then for all disjoint, inﬁnitesimal partitions, $B_1 ,\ldots, B_K$, the beta process is generated as follows, $H(B_k) \sim Beta(\alpha H_0 (B_k ), \alpha(1 − H_0(B_k )))$

with K → ∞ and $H_0(B_k)$ → 0 for $k = 1,\ldots,K$. This process is denoted $H \sim BP(\alpha H_0 )$."

This is the definition for a Dirichlet Process (DP):

If $X \sim DP(\alpha H_0)$ where $\alpha$ is a scalar and $H_0$ is a probability distribution, then for every finite partition $A_1,\ldots,A_K$ it follows that $(X(A_1),\ldots,X(A_K)) \sim DIR(\alpha H_0(A_1), \ldots,\alpha H_0(A_K))$.

So let's assume that I have $X\sim DP(\alpha H_0)$. Given any partition $B_1 ,\ldots, B_K$, and any $k = 1 \ldots K$, I can define a partition $A_1 = B_k, A_2 = \Omega -B_k$ and from the DP definition it follows that

$(X(A_1),X(A_2)) \sim DIR(\alpha H_0(A_1), \alpha H_0(A_2))$ which is equivalent to saying that $X(B_k) \sim Beta(\alpha H_0(B_k), \alpha(1-H_0(B_k)))$

hence $X\sim BP(\alpha H_0)$. Where is my mistake?

Answer by Gjergji Zaimi for Difference between Beta Process and Dirichlet process

2009-12-29T04:35:20Z

One cannot conclude $X\sim BP(\alpha H_0)$ just by knowing the marginal distribution of each $X(B_k)$, separately. Your calculation is not wrong as the univariate marginal distribution and conditional distribution of a Dirichlet distribution are Beta distributed.

In particular, in a Dirichlet process, samples correspond to the density function $$f(\theta)=\sum \beta_i \delta_{\theta_i}$$ (Here the $\beta_i$ are constructed as $\beta_i=\beta_i'\prod_{j < i}(1-\beta_j')$, and $\beta_j'\sim \text{Beta}(1,\alpha)=Y$)

While in a Beta process, given an infinitesimal partition $(B_1,...,B_K)$ with $K\to \infty$ and $H(B_k)\to 0$ the samples correspond to the density function $$H(B)=\sum \pi_i\delta_{B_i}$$ where $\pi_i\sim \text{Beta}(\alpha H_0(B_i),\alpha (1-H_0(B_i)))$

I hope you can see the difference. One more thing, the reason why the Dirichlet process is defined in terms of finitely dimensional distributions is because Kolmogorov extension theorem guarantees that it defines a stochastic process. Unfortunately the Beta process, does not verify the conditions of this theorem, and as a continuous time Levy process must be defined directly in the infinitesimal limit.

Answer by Jonathan for Difference between Beta Process and Dirichlet process

2009-12-29T08:43:14Z

I don't understand why you say that I can't conclude $X\sim BP(\alpha H_0)$. I showed above that if $X\sim DP(\alpha H_0)$ then for every partition $B_1...B_K$, $X(B_k)\sim Beta(\alpha H_0(B_k), \alpha(1-H_0(B_k)))$. By the definition of a Beta Process it follows that $X\sim BP(\alpha H_0)$. No?