# Difference between Beta Process and Dirichlet process

I'm trying to understand the definition of a Beta process, as given in the paper: www.ece.duke.edu/~lcarin/Paisley_BP-FA_ICML.pdf

The problem is that from the definition it follows that every Dirichlet process is also a beta process, which seems, ahm, wrong. Can you help me figure out what I don't understand?

This is the definition from the paper: "Let $H_0$ be a continuous probability measure and α a positive scalar. Then for all disjoint, inﬁnitesimal partitions, $B_1 ,\ldots, B_K$, the beta process is generated as follows, $H(B_k) \sim Beta(\alpha H_0 (B_k ), \alpha(1 − H_0(B_k )))$

with K → ∞ and $H_0(B_k)$ → 0 for $k = 1,\ldots,K$. This process is denoted $H \sim BP(\alpha H_0 )$."

This is the definition for a Dirichlet Process (DP):

If $X \sim DP(\alpha H_0)$ where $\alpha$ is a scalar and $H_0$ is a probability distribution, then for every finite partition $A_1,\ldots,A_K$ it follows that $(X(A_1),\ldots,X(A_K)) \sim DIR(\alpha H_0(A_1), \ldots,\alpha H_0(A_K))$.

So let's assume that I have $X\sim DP(\alpha H_0)$. Given any partition $B_1 ,\ldots, B_K$, and any $k = 1 \ldots K$, I can define a partition $A_1 = B_k, A_2 = \Omega -B_k$ and from the DP definition it follows that

$(X(A_1),X(A_2)) \sim DIR(\alpha H_0(A_1), \alpha H_0(A_2))$ which is equivalent to saying that $X(B_k) \sim Beta(\alpha H_0(B_k), \alpha(1-H_0(B_k)))$

hence $X\sim BP(\alpha H_0)$. Where is my mistake?

-

One cannot conclude $X\sim BP(\alpha H_0)$ just by knowing the marginal distribution of each $X(B_k)$, separately. Your calculation is not wrong as the univariate marginal distribution and conditional distribution of a Dirichlet distribution are Beta distributed.

In particular, in a Dirichlet process, samples correspond to the density function $$f(\theta)=\sum \beta_i \delta_{\theta_i}$$ (Here the $\beta_i$ are constructed as $\beta_i=\beta_i'\prod_{j < i}(1-\beta_j')$, and $\beta_j'\sim \text{Beta}(1,\alpha)=Y$)

While in a Beta process, given an infinitesimal partition $(B_1,...,B_K)$ with $K\to \infty$ and $H(B_k)\to 0$ the samples correspond to the density function $$H(B)=\sum \pi_i\delta_{B_i}$$ where $\pi_i\sim \text{Beta}(\alpha H_0(B_i),\alpha (1-H_0(B_i)))$

I hope you can see the difference. One more thing, the reason why the Dirichlet process is defined in terms of finitely dimensional distributions is because Kolmogorov extension theorem guarantees that it defines a stochastic process. Unfortunately the Beta process, does not verify the conditions of this theorem, and as a continuous time Levy process must be defined directly in the infinitesimal limit.

-
OK, I think that (thanks to your help) I understand the difference. Let me put it in my own words: The definition of the DP was basically a criterion: "When can we say that X is drawn from a DP? If at the level of every finite partition X is Dirichlet distributed". The definition of the BP is an (asymptotic) construction, not a criterion. It implies, for example, that the atomic weights, $\pi_i$ are drawn independently, unlike the DP (in which they must sum to 1). – Jonathan Dec 31 '09 at 3:41
By the way, regarding your remark on the Kolmogorov extension theorem, you can check this paper: books.nips.cc/papers/files/nips22/NIPS2009_0901.pdf – Jonathan Dec 31 '09 at 3:42

I don't understand why you say that I can't conclude $X\sim BP(\alpha H_0)$. I showed above that if $X\sim DP(\alpha H_0)$ then for every partition $B_1...B_K$, $X(B_k)\sim Beta(\alpha H_0(B_k), \alpha(1-H_0(B_k)))$. By the definition of a Beta Process it follows that $X\sim BP(\alpha H_0)$. No?

-
Jonathan: Oh, I see, you probably couldn't comment because you aren't recognized as the same user. – Jonas Meyer Dec 29 '09 at 9:14
Gjergji: I can't follow your reasoning. First, I don't see why $X(B_1)$ will depend on the partition that $B_1$ comes from. In both definitions (for DP and for BP) $X(B_1)$ only depends on $\alpha$ and $H_0(B_1)$. Specifically, regarding the partition you suggested $(B_1, B_2, \Omega-(B_1,B_2))$, I will argue that $X(B_1)$ is still Beta distributed. – Jonathan Dec 30 '09 at 0:30
I edited my post, hope it makes things clearer. – Gjergji Zaimi Dec 30 '09 at 6:38