General version of $d$-separation

Question

I find the $d$-separation criterion (see, e.g., Theorem 2 here; note however the preceding definition, which basically means we are treating discrete random variables) a really useful sufficient criterion for conditional independence of random variables. However, I am struggling to find a general version for arbitrary random variables, say with values in a Polish space (where conditional independence is defined via the respective disintegrations).

My question: Is there a version of $d$-separation that treats arbitrary Polish spaces valued random variables, or at least arbitrary $\mathbb{R}^d$ valued random variables?

To make the post self-contained: Let $\pi \in \mathcal{P}(S_1\times S_2 \times \dots \times S_N)$ be a Borel probability measure on the polish space $S_1 \times ... \times S_N$. Assume there is a directed acyclic graph with nodes $\{1, 2, \dots, N\}$, where without loss of generality the nodes are increasing in topological order (i.e., there is no edge pointing from $k$ to $l$ for $k > l$). Denote by $pa(k)$ the set of parents of a node $k$. Assume $\pi$ has a regular disintegration of the form $$ \pi = \pi_{1} \otimes \pi_{2, pa(2)} \otimes \pi_{3, pa(3)} \otimes \dots \otimes \pi_{N, pa(N)}, $$ where $\pi_{k, pa(k)}$ is a stochastic kernel mapping from $(\otimes_{j \in pa(k)} S_j)$ to $\mathcal{P}(S_k)$. Note that this disintegration naturally encodes certain conditional independencies. Indeed, if $(X_1, X_2, \dots, X_N) \sim \pi$ is a random variable with distribution $\pi$, the given disintegration means that $X_k$ is conditionally independent of $\{X_j : j \in \{1, ..., k-1\}\backslash pa(k)\}$ given $\{X_j : j \in pa(k)\}$.

On the other hand, this disintegration also implies many other conditional independences between the variables, and $d$-separation is a criterion to identify all of them (atleast for discrete spaces). The definition of $d$-separation is as follows: Take disjoint $A, B, C \subset \{1, \dots, N\}$. We say that $C$ $d$-separates $A$ and $B$, if there are no paths from a node $i \in A$ to a node $j \in B$ which is not blocked by a node in $C$. A path between nodes is blocked if either of the following criteria is satisfied:

1. there is a chain element $a \rightarrow b \leftarrow c$ contained in the path such that $b \not \in C$ and none of the descendants of $b$ is in $C$.
2. there is a chain element $a \rightarrow b \rightarrow c$ or $a \leftarrow b \rightarrow c$ contained in the path such that $b \in C$.

Then the fundamental result we want is that if $C$ $d$-separates $A$ and $B$, then $\{X_j : j\in A\}$ and $\{X_j : j \in B\}$ are conditionally independent given $\{X_j : j \in C\}$ (all under $\pi$, of course).

Answer 1

Yes, there are more general versions of the $d$-separation criterion, in particular also for standard Borel spaces (i.e. Polish spaces).

For completeness, the classical version of $d$-separation (i.e. on discrete random variables) has been generalized several times to the best of my knowledge.

1. $d$-separation for absolute continuous probability distributions

Lauritzen et al. (see Independence Properties of Directed Markov Fields) showed that the $d$-separation criterion holds for absolute continuous probability distributions with respect to a product measure. More specifically, let $\mathcal{H}_1, \ldots, \mathcal{H}_n$ be measurable spaces and $P$ a probability distribution on $\prod_{i=1}^{n} \mathcal{H}_i$ such that $$P(X_1 \in E_1, \ldots, X_n \in E_n) = \int_{x_1 \in E_1} \cdots \int_{x_n \in E_n} f(x_1,\ldots, x_n) \ \mu_1(\mathrm{d} x_1) \otimes \cdots \otimes \mu_n(\mathrm{d} x_n)$$ where $E_i$ is measurable, $f$ being a density and $\mu_i$ being a measure on $\mathcal{H}_i$. In this situation, a distribution $P$ is called compatible with a DAG $G$ (with vertices $\{1, \ldots, n\}$) if the density $f$ factorizes as $$f(x_1, \ldots, x_n) = \prod_{i=1}^{n} f(x_i | x_{\textsf{pa}(i)}) \quad \mu\textrm{-a.e.}$$ where $\textsf{pa}(i)$ is the set of parents of vertex $i$ and $f(x_i | x_{\textsf{pa}(i)})$ is the conditional density of $X_i$ given its parents.

Theorem 1 in the reference shows that the following two statements are equivalent

• $P$ is compatible with $G$
• $P$ satisfies the global Markov property, i.e. for every disjoint triple of subsets $A,B,C \subseteq \{1, \ldots, n\}$ such that $C$ d-separates $A$ and $B$ in $G$, we have that $$f(x_A, x_B, x_C) = f(x_A | x_C) \cdot f(x_B | x_C) \cdot f(x_C) \quad \mu\textrm{-a.e.},$$ i.e. $\{X_i: i\in A\}$ is cond. independent of $\{X_j: j \in B\}$ given $\{X_k: k\in C\}$.

2. $d$-separation for standard Borel spaces (i.e. Polish spaces)

A version of the $d$-separation criterion (with several other equivalent conditions) for standard Borel spaces has been proven in by P. Forré and J. Mooij (Markov Properties for Graphical Models with Cycles and Latent Variables). Your desired version is stated in Theorem 3.2.1. ($3. \Longleftrightarrow 6.$)

In this setting a probability distribution on standard Borel spaces $\prod_{i=1}^n \mathcal{H}_i$ given by $$P(X_1 \in E_1, \ldots, X_n \in E_n) = \int_{x_1 \in E_1} \cdots \int_{x_n \in E_n} P(\mathrm{d} x_1, \ldots, \mathrm{d} x_n)$$ with $E_{i} \in \mathcal{H}_i$ is compatible with a DAG $G$ if $$P(\mathrm{d} x_1, \ldots, \mathrm{d} x_n) = \prod_{i = 1}^n P(\mathrm{d} x_i | x_{\textsf{pa}(i)})$$ where $P(\mathrm{d} x_i | x_{\textsf{pa}(i)})$ is the regular conditional probability distribution of $X_i$ given its parents in $G$. In Theorem 3.2.1. ($3. \Longleftrightarrow 6.$) it is proven that the following two statements are equivalent:

• $P$ is compatible with $G$
• $P$ satisfies the global Markov property, i.e. for every disjoint triple of subsets $A, B, C \subseteq \{1,\ldots, n\}$ such that $A$ is $d$-separated from $B$ by $C$, we have that $\{X_i: i\in A\}$ is cond. independent of $\{X_j: j \in B\}$ given $\{X_k: k\in C\}$ where conditional independence is defined via regular conditional probabilities.

3. $d$-separation on Markov categories

As already mentioned in the question, conditional independence is defined via regular conditional probability distributions in all examples considered.

Recently, Tobias Fritz and me showed the $d$-separation criterion in the more abstract setting of Markov categories (see The $d$-Separation Criterion in Categorical Probability). There we have shown that whenever you can do an operation that behaves like Bayesian disintegration (i.e. whenever conditionals exist), the $d$-separation criterion applies.

This result includes the $d$-separation criterion on standard Borel spaces and finite probability distributions as specific instances. But it also includes non-probabilistic settings, where $d$-separation applies.

For example, it implies that the $d$-separation criterion also applies to finite possibilistic networks, i.e. networks where a conditional $f(x | a)$ specifies whether $x \in X$ is possible or impossible given $a \in A$ by setting $f(x | a) = 1$ or $f(x|a) = 0$.

Answer 2

It seems that Theorem 2 in the linked paper readily extends to variables taking values in polish spaces. The needed observation is that if the underlying random variables $X_1,\ldots,X_n$ take values in a Polish space $\Upsilon$, then the following equivalence holds:

$X_J$ and $X_K$ are conditionally independent given $X_L$ iff for all Borel functions $f:\Upsilon \to {\mathbb Z}$ with finite range, $f(X_J)$ and $f(X_K)$ are conditionally independent given the $\sigma$-field determined by $X_L$.

Here $f(X_L)$ is shorthand for the sequence $\{f(x_\ell)\}_{\ell \in L}$.