Return to Answer

A way to enrich $\mathsf{Stoch}$ could be as follows.

Let $X$ be a measurable space and let $PX$ be the space of probability measures over $X$. We can equip $PX$ with the total variational distance, which is given by the following equivalent forms. $$ d(p,q) \;=\; \sup_{A\subseteq X} |p(A) - q(A)| \;=\; \sup_{f:X\to[0,1]} \left| \int_X f\, dp - \int_X f\; dq \right| . $$ Above, and in what follows, the subsets $A\subseteq X$ and the functions $f:X\to\Bbb{R}$ are assumed measurable.

Now first of all let $f:X \to Y$ be a measurable (deterministic) function. Denote by $f_*:PX\to PY$ the pushforward of measures (for category theorists, the action of the functor $P$ on the morphisms). We have \begin{align*} d( f_*p, f_*q ) \;&=\; \sup_{B\subseteq Y} |f_*p(B) - f_*q(B)| \\\ \;&=\; \sup_{B\subseteq Y} |p(f^{-1}(B)) - q(f^{-1}(B))| \\\ \;&\le\; \sup_{A\subseteq X} |p(A) - q(A)| \\\ \;&=\; d( p, q ) , \end{align*} which means that $f_*:PX\to PY$ is short for the metric just defined. This will be of use later.

We can equip now the sets $\mathsf{Stoch}(X,Y)$ with the "product" or "$L^\infty$" metric construction induced by the one on $PY$, as follows. We use the following notation, a stochastic map $k:X\to PY$ evaluated at a point $x\in X$ gives the measure $k_x\in PY$. Now for stochastic maps $k,h:X\to PY$, we set $$ d(k,h) \;=\; \sup_{x\in X} d\big( k_x, h_y \big) . $$ This number is bounded by $1$. (Thanks to Martin Hairer for pointing this out, see the comments.)

We now have to prove that the (Kleisli) composition of $\mathsf{Stoch}$ is short. Given $h:X \to PY$ and $l: Y \to PZ$, their Kleisli composition, which we denote by $lh$, is given by the Chapman-Kolmogorov formula. Explicitly, for every measurable $C \subseteq Z$, we have $$ lh_x(C) \;=\; \int_{PZ} q(C) \,d(l_*h_x)(q) \;=\; \int_{Y} l_y(C) \,dh_x(y) . $$

Now, to see that postcomposition is short, let $h,k:X \to PY$ and $l: Y \to PZ$. We have \begin{align*} d( lh, lk ) \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} | lh_x(C) - lk_x(C) | \\\ \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} \left| \int_{PZ} q(C) \,d(l_*h_x)(q) - \int_{PZ} q(C) \,d(l_*k_x)(q) \right| \\\ \;&\le\; \sup_{x\in X} d\big( l_*h_x , l_*k_x \big) \\\ \;&\le\; \sup_{x\in X} d( h_x , k_x ) \\\ \;&=\; d( h, k ) . \end{align*} (We used the fact that the evaluation map $PX\to \Bbb{R}$ given by $p\mapsto p(A)$ for a measurable set $A\subseteq X$ is measurable for the $\sigma$-algebra on $PX$ in the construction of the Giry monad.)

To see that precomposition is short, let also $g: W \to PX$. Then \begin{align*} d( hg, kg ) \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} | hg_w(B) - kg_w(B) | \\\ \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} \left| \int_{X} h_x(B) \,dg_w(x) - \int_{X} k_x(B) \,dg_w(x) \right| \\\ \;&\le\; \sup_{w\in W} \int_{X} \sup_{B\subseteq Y} | h_x(B) - k_x(B) | \,d(g_w)(x) \\\ \;&=\; \sup_{w\in W} \int_{X} d( h_x, k_x ) \,d(g_w)(x) \\\ \;&\le\; d( h_x, k_x ) . \end{align*}

Consider now the set $\mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z)$. We can equip it either with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; \max\{ d(h_1,h_2), d(l_1,l_2) \}, $$ which corresponds to the metric of the cartesian (categorical) product of $\mathsf{Met}$, or with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; d(h_1,h_2) + d(l_1,l_2) , $$ which is the metric of the closed monoidal product of $\mathsf{Met}$ as an (i.e. enhibiting a hom-tensor adjunction).

For either choice, the map $\circ: \mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z) \to \mathsf{Stoch}(X,Z)$ given by composition is short. This gives the desired enrichment.

By the way, if we want talk about diagrams "commuting up to $\varepsilon$", then by definition of the metric, $\varepsilon$ has to be a uniform bound. For example, for the diagram $\require{AMScd}$ \begin{CD} A @>f>> B\\ @V m V V @VV n V\\ C @>>g> D \end{CD} to commute up to $\varepsilon$, in the sense that $d(n\circ f,g\circ m)\lt\varepsilon$, we need that $$ \sup_{a\in A} d\big(n(f(a)),g(m(a))\big) \;\lt\;\varepsilon, $$ where the distance above is the one of $D$. For that to happen, $\varepsilon$ needs to be independent of $a$, which is quite a strong condition in practice (if $A$ is not compact).

A way to enrich $\mathsf{Stoch}$ could be as follows.

Let $X$ be a measurable space and let $PX$ be the space of probability measures over $X$. We can equip $PX$ with the total variational distance, which is given by the following equivalent forms. $$ d(p,q) \;=\; \sup_{A\subseteq X} |p(A) - q(A)| \;=\; \sup_{f:X\to[0,1]} \left| \int_X f\, dp - \int_X f\; dq \right| . $$ Above, and in what follows, the subsets $A\subseteq X$ and the functions $f:X\to\Bbb{R}$ are assumed measurable.

Now first of all let $f:X \to Y$ be a measurable (deterministic) function. Denote by $f_*:PX\to PY$ the pushforward of measures (for category theorists, the action of the functor $P$ on the morphisms). We have \begin{align*} d( f_*p, f_*q ) \;&=\; \sup_{B\subseteq Y} |f_*p(B) - f_*q(B)| \\\ \;&=\; \sup_{B\subseteq Y} |p(f^{-1}(B)) - q(f^{-1}(B))| \\\ \;&\le\; \sup_{A\subseteq X} |p(A) - q(A)| \\\ \;&=\; d( p, q ) , \end{align*} which means that $f_*:PX\to PY$ is short for the metric just defined. This will be of use later.

We can equip now the sets $\mathsf{Stoch}(X,Y)$ with the "product" or "$L^\infty$" metric construction induced by the one on $PY$, as follows. We use the following notation, a stochastic map $k:X\to PY$ evaluated at a point $x\in X$ gives the measure $k_x\in PY$. Now for stochastic maps $k,h:X\to PY$, we set $$ d(k,h) \;=\; \sup_{x\in X} d\big( k_x, h_y \big) . $$ This number is bounded by $1$. (Thanks to Martin Hairer for pointing this out, see the comments.)

We now have to prove that the (Kleisli) composition of $\mathsf{Stoch}$ is short. Given $h:X \to PY$ and $l: Y \to PZ$, their Kleisli composition, which we denote by $lh$, is given by the Chapman-Kolmogorov formula. Explicitly, for every measurable $C \subseteq Z$, we have $$ lh_x(C) \;=\; \int_{PZ} q(C) \,d(l_*h_x)(q) \;=\; \int_{Y} l_y(C) \,dh_x(y) . $$

Now, to see that postcomposition is short, let $h,k:X \to PY$ and $l: Y \to PZ$. We have \begin{align*} d( lh, lk ) \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} | lh_x(C) - lk_x(C) | \\\ \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} \left| \int_{PZ} q(C) \,d(l_*h_x)(q) - \int_{PZ} q(C) \,d(l_*k_x)(q) \right| \\\ \;&\le\; \sup_{x\in X} d\big( l_*h_x , l_*k_x \big) \\\ \;&\le\; \sup_{x\in X} d( h_x , k_x ) \\\ \;&=\; d( h, k ) . \end{align*} (We used the fact that the evaluation map $PX\to \Bbb{R}$ given by $p\mapsto p(A)$ for a measurable set $A\subseteq X$ is measurable for the $\sigma$-algebra on $PX$ in the construction of the Giry monad.)

To see that precomposition is short, let also $g: W \to PX$. Then \begin{align*} d( hg, kg ) \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} | hg_w(B) - kg_w(B) | \\\ \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} \left| \int_{X} h_x(B) \,dg_w(x) - \int_{X} k_x(B) \,dg_w(x) \right| \\\ \;&\le\; \sup_{w\in W} \int_{X} \sup_{B\subseteq Y} | h_x(B) - k_x(B) | \,d(g_w)(x) \\\ \;&=\; \sup_{w\in W} \int_{X} d( h_x, k_x ) \,d(g_w)(x) \\\ \;&\le\; d( h_x, k_x ) . \end{align*}

Consider now the set $\mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z)$. We can equip it either with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; \max\{ d(h_1,h_2), d(l_1,l_2) \}, $$ which corresponds to the metric of the cartesian (categorical) product of $\mathsf{Met}$, or with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; d(h_1,h_2) + d(l_1,l_2) , $$ which is the metric of the closed monoidal product of $\mathsf{Met}$ as an (i.e. exhibiting a hom-tensor adjunction).

For either choice, the map $\circ: \mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z) \to \mathsf{Stoch}(X,Z)$ given by composition is short. This gives the desired enrichment.

By the way, if we want talk about diagrams "commuting up to $\varepsilon$", then by definition of the metric, $\varepsilon$ has to be a uniform bound. For example, for the diagram $\require{AMScd}$ \begin{CD} A @>f>> B\\ @V m V V @VV n V\\ C @>>g> D \end{CD} to commute up to $\varepsilon$, in the sense that $d(n\circ f,g\circ m)\lt\varepsilon$, we need that $$ \sup_{a\in A} d\big(n(f(a)),g(m(a))\big) \;\lt\;\varepsilon, $$ where the distance above is the one of $D$. For that to happen, $\varepsilon$ needs to be independent of $a$, which is quite a strong condition in practice (if $A$ is not compact).

A way to enrich $\mathsf{Stoch}$ could be as follows.

Let $X$ be a measurable space and let $PX$ be the space of probability measures over $X$. We can equip $PX$ with the total variational distance, which is given by the following equivalent forms. $$ d(p,q) \;=\; \sup_{A\subseteq X} |p(A) - q(A)| \;=\; \sup_{f:X\to[0,1]} \left| \int_X f\, dp - \int_X f\; dq \right| . $$ Above, and in what follows, the subsets $A\subseteq X$ and the functions $f:X\to\Bbb{R}$ are assumed measurable.

Now first of all let $f:X \to Y$ be a measurable (deterministic) function. Denote by $f_*:PX\to PY$ the pushforward of measures (for category theorists, the action of the functor $P$ on the morphisms). We have \begin{align*} d( f_*p, f_*q ) \;&=\; \sup_{B\subseteq Y} |f_*p(B) - f_*q(B)| \\\ \;&=\; \sup_{B\subseteq Y} |p(f^{-1}(B)) - q(f^{-1}(B))| \\\ \;&\le\; \sup_{A\subseteq X} |p(A) - q(A)| \\\ \;&=\; d( p, q ) , \end{align*} which means that $f_*:PX\to PY$ is short for the metric just defined. This will be of use later.

We can equip now the sets $\mathsf{Stoch}(X,Y)$ with the "product" or "$L^\infty$" metric construction induced by the one on $PY$, as follows. We use the following notation, a stochastic map $k:X\to PY$ evaluated at a point $x\in X$ gives the measure $k_x\in PY$. Now for stochastic maps $k,h:X\to PY$, we set $$ d(k,h) \;=\; \sup_{x\in X} d\big( k_x, h_y \big) . $$ This number is bounded by $1$. (Thanks to Martin Hairer for pointing this out, see the comments.)

We now have to prove that the (Kleisli) composition of $\mathsf{Stoch}$ is short. Given $h:X \to PY$ and $l: Y \to PZ$, their Kleisli composition, which we denote by $lh$, is given by the Chapman-Kolmogorov formula. Explicitly, for every measurable $C \subseteq Z$, we have $$ lh_x(C) \;=\; \int_{PZ} q(C) \,d(l_*h_x)(q) \;=\; \int_{Y} l_y(C) \,dh_x(y) . $$

Now, to see that postcomposition is short, let $h,k:X \to PY$ and $l: Y \to PZ$. We have \begin{align*} d( lh, lk ) \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} | lh_x(C) - lk_x(C) | \\\ \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} \left| \int_{PZ} q(C) \,d(l_*h_x)(q) - \int_{PZ} q(C) \,d(l_*k_x)(q) \right| \\\ \;&\le\; \sup_{x\in X} d\big( l_*h_x , l_*k_x \big) \\\ \;&\le\; \sup_{x\in X} d( h_x , k_x ) \\\ \;&=\; d( h, k ) . \end{align*} (We used the fact that the evaluation map $PX\to \Bbb{R}$ given by $p\mapsto p(A)$ for a measurable set $A\subseteq X$ is measurable for the $\sigma$-algebra on $PX$ in the construction of the Giry monad.)

To see that precomposition is short, let also $g: W \to PX$. Then \begin{align*} d( hg, kg ) \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} | hg_w(B) - kg_w(B) | \\\ \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} \left| \int_{X} h_x(B) \,dg_w(x) - \int_{X} k_x(B) \,dg_w(x) \right| \\\ \;&\le\; \sup_{w\in W} \int_{X} \sup_{B\subseteq Y} | h_x(B) - k_x(B) | \,d(g_w)(x) \\\ \;&=\; \sup_{w\in W} \int_{X} d( h_x, k_x ) \,d(g_w)(x) \\\ \;&\le\; d( h_x, k_x ) . \end{align*}

Consider now the set $\mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z)$. We can equip it either with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; \max\{ d(h_1,h_2), d(l_1,l_2) \}, $$ which corresponds to the metric of the cartesian (categorical) product of $\mathsf{Met}$, or with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; d(h_1,h_2) + d(l_1,l_2) , $$ which is the metric of the closed monoidal product of $\mathsf{Met}$ as an (i.e. enhibiting a hom-tensor adjunction).

For either choice, the map $\circ: \mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z) \to \mathsf{Stoch}(X,Z)$ given by composition is short. This gives the desired enrichment.

By the way, if we want talk about diagrams "commuting up to $\varepsilon$", then by definition of the metric, $\varepsilon$ has to ba a uniform bound. For example, for the diagram $\require{AMScd}$ \begin{CD} A @>f>> B\\ @V m V V @VV n V\\ C @>>g> D \end{CD} to commute up to $\varepsilon$, in the sense that $d(n\circ f,g\circ m)\lt\varepsilon$, we need that $$ \sup_{a\in A} d\big(n(f(a)),g(m(a))\big) \;\lt\;\varepsilon, $$ where the distance above is the one of $D$. For that to happen, $\varepsilon$ needs to be independent of $a$, which is quite a strong condition in practice (if $A$ is not compact).

A way to enrich $\mathsf{Stoch}$ could be as follows.

Let $X$ be a measurable space and let $PX$ be the space of probability measures over $X$. We can equip $PX$ with the total variational distance, which is given by the following equivalent forms. $$ d(p,q) \;=\; \sup_{A\subseteq X} |p(A) - q(A)| \;=\; \sup_{f:X\to[0,1]} \left| \int_X f\, dp - \int_X f\; dq \right| . $$ Above, and in what follows, the subsets $A\subseteq X$ and the functions $f:X\to\Bbb{R}$ are assumed measurable.

Now first of all let $f:X \to Y$ be a measurable (deterministic) function. Denote by $f_*:PX\to PY$ the pushforward of measures (for category theorists, the action of the functor $P$ on the morphisms). We have \begin{align*} d( f_*p, f_*q ) \;&=\; \sup_{B\subseteq Y} |f_*p(B) - f_*q(B)| \\\ \;&=\; \sup_{B\subseteq Y} |p(f^{-1}(B)) - q(f^{-1}(B))| \\\ \;&\le\; \sup_{A\subseteq X} |p(A) - q(A)| \\\ \;&=\; d( p, q ) , \end{align*} which means that $f_*:PX\to PY$ is short for the metric just defined. This will be of use later.

We can equip now the sets $\mathsf{Stoch}(X,Y)$ with the "product" or "$L^\infty$" metric construction induced by the one on $PY$, as follows. We use the following notation, a stochastic map $k:X\to PY$ evaluated at a point $x\in X$ gives the measure $k_x\in PY$. Now for stochastic maps $k,h:X\to PY$, we set $$ d(k,h) \;=\; \sup_{x\in X} d\big( k_x, h_y \big) . $$ This number is bounded by $1$. (Thanks to Martin Hairer for pointing this out, see the comments.)

We now have to prove that the (Kleisli) composition of $\mathsf{Stoch}$ is short. Given $h:X \to PY$ and $l: Y \to PZ$, their Kleisli composition, which we denote by $lh$, is given by the Chapman-Kolmogorov formula. Explicitly, for every measurable $C \subseteq Z$, we have $$ lh_x(C) \;=\; \int_{PZ} q(C) \,d(l_*h_x)(q) \;=\; \int_{Y} l_y(C) \,dh_x(y) . $$

Now, to see that postcomposition is short, let $h,k:X \to PY$ and $l: Y \to PZ$. We have \begin{align*} d( lh, lk ) \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} | lh_x(C) - lk_x(C) | \\\ \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} \left| \int_{PZ} q(C) \,d(l_*h_x)(q) - \int_{PZ} q(C) \,d(l_*k_x)(q) \right| \\\ \;&\le\; \sup_{x\in X} d\big( l_*h_x , l_*k_x \big) \\\ \;&\le\; \sup_{x\in X} d( h_x , k_x ) \\\ \;&=\; d( h, k ) . \end{align*} (We used the fact that the evaluation map $PX\to \Bbb{R}$ given by $p\mapsto p(A)$ for a measurable set $A\subseteq X$ is measurable for the $\sigma$-algebra on $PX$ in the construction of the Giry monad.)

To see that precomposition is short, let also $g: W \to PX$. Then \begin{align*} d( hg, kg ) \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} | hg_w(B) - kg_w(B) | \\\ \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} \left| \int_{X} h_x(B) \,dg_w(x) - \int_{X} k_x(B) \,dg_w(x) \right| \\\ \;&\le\; \sup_{w\in W} \int_{X} \sup_{B\subseteq Y} | h_x(B) - k_x(B) | \,d(g_w)(x) \\\ \;&=\; \sup_{w\in W} \int_{X} d( h_x, k_x ) \,d(g_w)(x) \\\ \;&\le\; d( h_x, k_x ) . \end{align*}

Consider now the set $\mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z)$. We can equip it either with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; \max\{ d(h_1,h_2), d(l_1,l_2) \}, $$ which corresponds to the metric of the cartesian (categorical) product of $\mathsf{Met}$, or with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; d(h_1,h_2) + d(l_1,l_2) , $$ which is the metric of the closed monoidal product of $\mathsf{Met}$ as an (i.e. enhibiting a hom-tensor adjunction).

For either choice, the map $\circ: \mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z) \to \mathsf{Stoch}(X,Z)$ given by composition is short. This gives the desired enrichment.

By the way, if we want talk about diagrams "commuting up to $\varepsilon$", then by definition of the metric, $\varepsilon$ has to be a uniform bound. For example, for the diagram $\require{AMScd}$ \begin{CD} A @>f>> B\\ @V m V V @VV n V\\ C @>>g> D \end{CD} to commute up to $\varepsilon$, in the sense that $d(n\circ f,g\circ m)\lt\varepsilon$, we need that $$ \sup_{a\in A} d\big(n(f(a)),g(m(a))\big) \;\lt\;\varepsilon, $$ where the distance above is the one of $D$. For that to happen, $\varepsilon$ needs to be independent of $a$, which is quite a strong condition in practice (if $A$ is not compact).

A way to enrich $\mathsf{Stoch}$ could be as follows, allowing infinite distances (as is standard in category theory).

Let $X$ be a measurable space and let $PX$ be the space of probability measures over $X$. We can equip $PX$ with the total variational distance, which is given by the following equivalent forms. $$ d(p,q) \;=\; \sup_{A\subseteq X} |p(A) - q(A)| \;=\; \sup_{f:X\to\Bbb{R}} \left| \int_X f\, dp - \int_X f\; dq \right| . $$ Above, and in what follows, the subsets $A\subseteq X$ and the functions $f:X\to\Bbb{R}$ are assumed measurable.

Now first of all let $f:X \to Y$ be a measurable (deterministic) function. Denote by $f_*:PX\to PY$ the pushforward of measures (for category theorists, the action of the functor $P$ on the morphisms). We have \begin{align*} d( f_*p, f_*q ) \;&=\; \sup_{B\subseteq Y} |f_*p(B) - f_*q(B)| \\\ \;&=\; \sup_{B\subseteq Y} |p(f^{-1}(B)) - q(f^{-1}(B))| \\\ \;&\le\; \sup_{A\subseteq X} |p(A) - q(A)| \\\ \;&=\; d( p, q ) , \end{align*} which means that $f_*:PX\to PY$ is short for the metric just defined. This will be of use later.

We can equip now the sets $\mathsf{Stoch}(X,Y)$ with the "product" or "$L^\infty$" metric construction induced by the one on $PY$, as follows. We use the following notation, a stochastic map $k:X\to PY$ evaluated at a point $x\in X$ gives the measure $k_x\in PY$. Now for stochastic maps $k,h:X\to PY$, we set $$ d(k,h) \;=\; \sup_{x\in X} d\big( k_x, h_y \big) . $$ This supremum may be infinite, this is why we have to allow infinite distances.

We now have to prove that the (Kleisli) composition of $\mathsf{Stoch}$ is short. Given $h:X \to PY$ and $l: Y \to PZ$, their Kleisli composition, which we denote by $lh$, is given by the Chapman-Kolmogorov formula. Explicitly, for every measurable $C \subseteq Z$, we have $$ lh_x(C) \;=\; \int_{PZ} q(C) \,d(l_*h_x)(q) \;=\; \int_{Y} l_y(C) \,dh_x(y) . $$

Now, to see that postcomposition is short, let $h,k:X \to PY$ and $l: Y \to PZ$. We have \begin{align*} d( lh, lk ) \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} | lh_x(C) - lk_x(C) | \\\ \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} \left| \int_{PZ} q(C) \,d(l_*h_x)(q) - \int_{PZ} q(C) \,d(l_*k_x)(q) \right| \\\ \;&\le\; \sup_{x\in X} d\big( l_*h_x , l_*k_x \big) \\\ \;&\le\; \sup_{x\in X} d( h_x , k_x ) \\\ \;&=\; d( h, k ) . \end{align*} (We used the fact that the evaluation map $PX\to \Bbb{R}$ given by $p\mapsto p(A)$ for a measurable set $A\subseteq X$ is measurable for the $\sigma$-algebra on $PX$ in the construction of the Giry monad.)

To see that precomposition is short, let also $g: W \to PX$. Then \begin{align*} d( hg, kg ) \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} | hg_w(B) - kg_w(B) | \\\ \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} \left| \int_{X} h_x(B) \,dg_w(x) - \int_{X} k_x(B) \,dg_w(x) \right| \\\ \;&\le\; \sup_{w\in W} \int_{X} \sup_{B\subseteq Y} | h_x(B) - k_x(B) | \,d(g_w)(x) \\\ \;&=\; \sup_{w\in W} \int_{X} d( h_x, k_x ) \,d(g_w)(x) \\\ \;&\le\; d( h_x, k_x ) . \end{align*}

Consider now the set $\mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z)$. We can equip it either with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; \max\{ d(h_1,h_2), d(l_1,l_2) \}, $$ which corresponds to the metric of the cartesian (categorical) product of $\mathsf{Met}$, or with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; d(h_1,h_2) + d(l_1,l_2) , $$ which is the metric of the closed monoidal product of $\mathsf{Met}$ as an (i.e. enhibiting a hom-tensor adjunction).

For either choice, the map $\circ: \mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z) \to \mathsf{Stoch}(X,Z)$ given by composition is short. This gives the desired enrichment (with possibly infinite distances).

By the way, if we want talk about diagrams "commuting up to $\varepsilon$", then by definition of the metric, $\varepsilon$ has to ba a uniform bound. For example, for the diagram $\require{AMScd}$ \begin{CD} A @>f>> B\\ @V m V V @VV n V\\ C @>>g> D \end{CD} to commute up to $\varepsilon$, in the sense that $d(n\circ f,g\circ m)\lt\varepsilon$, we need that $$ \sup_{a\in A} d\big(n(f(a)),g(m(a))\big) \;\lt\;\varepsilon, $$ where the distance above is the one of $D$. For that to happen, $\varepsilon$ needs to be independent of $a$, which is quite a strong condition in practice (if $A$ is not compact).

A way to enrich $\mathsf{Stoch}$ could be as follows.

Let $X$ be a measurable space and let $PX$ be the space of probability measures over $X$. We can equip $PX$ with the total variational distance, which is given by the following equivalent forms. $$ d(p,q) \;=\; \sup_{A\subseteq X} |p(A) - q(A)| \;=\; \sup_{f:X\to[0,1]} \left| \int_X f\, dp - \int_X f\; dq \right| . $$ Above, and in what follows, the subsets $A\subseteq X$ and the functions $f:X\to\Bbb{R}$ are assumed measurable.

Now first of all let $f:X \to Y$ be a measurable (deterministic) function. Denote by $f_*:PX\to PY$ the pushforward of measures (for category theorists, the action of the functor $P$ on the morphisms). We have \begin{align*} d( f_*p, f_*q ) \;&=\; \sup_{B\subseteq Y} |f_*p(B) - f_*q(B)| \\\ \;&=\; \sup_{B\subseteq Y} |p(f^{-1}(B)) - q(f^{-1}(B))| \\\ \;&\le\; \sup_{A\subseteq X} |p(A) - q(A)| \\\ \;&=\; d( p, q ) , \end{align*} which means that $f_*:PX\to PY$ is short for the metric just defined. This will be of use later.

We can equip now the sets $\mathsf{Stoch}(X,Y)$ with the "product" or "$L^\infty$" metric construction induced by the one on $PY$, as follows. We use the following notation, a stochastic map $k:X\to PY$ evaluated at a point $x\in X$ gives the measure $k_x\in PY$. Now for stochastic maps $k,h:X\to PY$, we set $$ d(k,h) \;=\; \sup_{x\in X} d\big( k_x, h_y \big) . $$ This number is bounded by $1$. (Thanks to Martin Hairer for pointing this out, see the comments.)

We now have to prove that the (Kleisli) composition of $\mathsf{Stoch}$ is short. Given $h:X \to PY$ and $l: Y \to PZ$, their Kleisli composition, which we denote by $lh$, is given by the Chapman-Kolmogorov formula. Explicitly, for every measurable $C \subseteq Z$, we have $$ lh_x(C) \;=\; \int_{PZ} q(C) \,d(l_*h_x)(q) \;=\; \int_{Y} l_y(C) \,dh_x(y) . $$

Now, to see that postcomposition is short, let $h,k:X \to PY$ and $l: Y \to PZ$. We have \begin{align*} d( lh, lk ) \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} | lh_x(C) - lk_x(C) | \\\ \;&=\; \sup_{x\in X} \sup_{C\subseteq Z} \left| \int_{PZ} q(C) \,d(l_*h_x)(q) - \int_{PZ} q(C) \,d(l_*k_x)(q) \right| \\\ \;&\le\; \sup_{x\in X} d\big( l_*h_x , l_*k_x \big) \\\ \;&\le\; \sup_{x\in X} d( h_x , k_x ) \\\ \;&=\; d( h, k ) . \end{align*} (We used the fact that the evaluation map $PX\to \Bbb{R}$ given by $p\mapsto p(A)$ for a measurable set $A\subseteq X$ is measurable for the $\sigma$-algebra on $PX$ in the construction of the Giry monad.)

To see that precomposition is short, let also $g: W \to PX$. Then \begin{align*} d( hg, kg ) \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} | hg_w(B) - kg_w(B) | \\\ \;&=\; \sup_{w\in W} \sup_{B\subseteq Y} \left| \int_{X} h_x(B) \,dg_w(x) - \int_{X} k_x(B) \,dg_w(x) \right| \\\ \;&\le\; \sup_{w\in W} \int_{X} \sup_{B\subseteq Y} | h_x(B) - k_x(B) | \,d(g_w)(x) \\\ \;&=\; \sup_{w\in W} \int_{X} d( h_x, k_x ) \,d(g_w)(x) \\\ \;&\le\; d( h_x, k_x ) . \end{align*}

Consider now the set $\mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z)$. We can equip it either with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; \max\{ d(h_1,h_2), d(l_1,l_2) \}, $$ which corresponds to the metric of the cartesian (categorical) product of $\mathsf{Met}$, or with the metric $$ d \big( (h_1,l_1), (h_2,l_2) \big) \;=\; d(h_1,h_2) + d(l_1,l_2) , $$ which is the metric of the closed monoidal product of $\mathsf{Met}$ as an (i.e. enhibiting a hom-tensor adjunction).

For either choice, the map $\circ: \mathsf{Stoch}(X,Y)\times\mathsf{Stoch}(Y,Z) \to \mathsf{Stoch}(X,Z)$ given by composition is short. This gives the desired enrichment.

By the way, if we want talk about diagrams "commuting up to $\varepsilon$", then by definition of the metric, $\varepsilon$ has to ba a uniform bound. For example, for the diagram $\require{AMScd}$ \begin{CD} A @>f>> B\\ @V m V V @VV n V\\ C @>>g> D \end{CD} to commute up to $\varepsilon$, in the sense that $d(n\circ f,g\circ m)\lt\varepsilon$, we need that $$ \sup_{a\in A} d\big(n(f(a)),g(m(a))\big) \;\lt\;\varepsilon, $$ where the distance above is the one of $D$. For that to happen, $\varepsilon$ needs to be independent of $a$, which is quite a strong condition in practice (if $A$ is not compact).