Note: B-variables were called streams in a previous version -> you won't understand the comments otherwise
Definition of $B$-variables
Theorem: Let $l_1\leq \dots\leq l_n$ be the lengths of a set of binary strings. If $\sum 2^{-l_i}\leq 1$, then we can hope no string is the prefix of an other in the set.
Proof: First, we notice that $1-\sum2^{-l_i}$ is a multiple of $2^{-l_n}$, such that adding $2^{l_{n}}-\sum2^{l_{n}-l_{i}}$ times $l_{n}$ to our set of lengths we get $\sum 2^{-l_i} = 1$. Then, if we prune an infinite binary tree at depths $l_1,\dots,l_n$, the set of paths from the root to the leafs described using left/right instructions will form a valid prefix-distinct set of binary strings.
Now, given a finite probability distribution $p$ and noting $l_i=\left\lceil -\log_{2}p_i\right\rceil$, you can check $\sum 2^{l_i}\leq 1$.
Given two finite random variables $X$ and $Y$, we can look at the family of distributions $(p_{X|Y=y})_{y\in \text{Im}(Y)}$ and construct the corresponding sets of binary strings using the $(l_i)$ defined above.
Then for each $(x, y)$ in $\text{Im}(X)\times \text{Im}(Y)$ we have an associated binary string (this is not injective). By drawing strings using $X$ and $Y$ in this way, we get a new random variable noted $B_{X|Y}$.
Similarly, we can look at $X$ and $B_{X|Y}$ and form the variable $B_{X|B_{X|Y}}$. To avoid nesting indices and to improve the notation we will omit the $B$ if we already are inside a $B$-variable such that $B_{X|(X|Y)} = B_{X|B_{X|Y}}$.
Given two finite random variables $X$ and $Y$, we can also consider the joint distribution $p_{X,Y}$. We can associate a string to each $(x, y)$ in $\text{Im}(X)\times \text{Im}(Y)$ using this distribution and the usual $(l_i)$ and note the new random variable $B_{XY}$.
Finally, we can interpret $n$ independent draws of a random variable $X$ as the single draw of a variable with values in $\text{Im}(X)^n$ and we will note such variable $X^n$.
We generalize this notion differently to a $B$-variable: its $n^{th}$ power is obtained by taking the $n^{th}$ powers of the non-$B$ variables composing it (recursively).
For example,
$$B_{X|(X|Y)}^ n = B_{X|B_{X|Y}}^n=B_{X^n|B_{X^n|Y^n}}$$
$B$-variables form a category
Using the right morphism definition, $B$-variables form a category $\textbf{Bvar}$.
A morphism of $B$-variables $X\to Y$ is given by two sequences of surjections $(e_n)$ and $(d_n)$ such that,
$$\lim_{n\to \infty} Pr(e_n(X^n)=d_n(Y^n)) = 1 \tag{*}$$
and, using Landau's notation,
$$H(d_n(Y^n))=H(Y^n)+o(n) \tag{**}$$
where $H$ denotes Shannon's entropy.
To compose morphisms, we use the right inverses $(d_n^{-1})$ such that:
$$(e^{\prime}_n,d^{\prime}_n)\circ (e_n, d_n) = (e^{\prime}_n \circ d_n^{-1}\circ e_n, d^{\prime}_n)$$
Here, $(^{**})$ guarantees $(d_n^{-1})$ are "good-enough left inverses" to preserve $(^*)$ for the composite.
We must quotient the sequences $(e_n)$ and $(d_n)$ by $\lim H(e_n(X^n))/n$ and $\lim H(d_n(Y^n))/n$ to get identities (otherwise we only get right-identities -> thanks @Gro-Tsen).
PS: The codomains of $(e_n)$ and $(d_n)$ must be the same because of $(^*)$ and must grow large enough with $n$ to satisfy $(^{**})$.
Properties of $\textbf{Bvar}$
Given two finite random variables $X$ and $Y$, we have: $${B_{X}}\to{B_{X|Y}}\to0$$
And we can define the product: $${B_{X}}\times {B_{Y}}={B_{XY}}={B_{X(Y|X)}}={B_{Y(X|Y)}}$$ The coproduct is given by: $${B_{X}}+ {B_{Y}}={B_{XY|(X|Y)(Y|X)}}={B_{XY|(Y|X)(X|Y)}}$$
The product corresponds to the joint entropy and the coproduct to the mutual information.
Shannon's channel coding theorem is equivalent to proving the existence of coproducts in $\textbf{Bvar}$.
An entropy is a forgetful functor from $\textbf{Bvar}$ to $(\mathbb{R}^+,\leq)$.
Disclaimer: I'm a software engineer without pure math education barely knowing some category theory. This seemed nice but I don't know what category theorist usually look for when finding a new category, hence my post here. What else can be said about $\bf{Bvar}$?
