I am a little reticent to do this. This is just an outline that is based heavily on the BFK paper
you cited in the question, and what seems to be your observation. Maybe I am just repeating back
to you what you are thinking.
To start with $SL_2$ is the restricted dual of $U(sl_2)$ that is the linear functionals that factor through a finite dimensional representation. You can also think of $SL_2$ as the coordinate ring
of $SL_2\mathbb{C}$. There is a linear functional $\mu:SL_2\rightarrow \mathbb{C}$ given by restricting the function to $SU(2)$ and integrating against Haar measure. Even though its analytic
its really algebraic, as Weyl orthogonality gives a complete set of rules for evaluating it.
$SL_2$ is a Hopf algebra, which you will need for the next construction.
Taking the tensor product of $\mu$ with itself $k$-times you get $\mu^{\otimes k}:SL_2^{\otimes k}\rightarrow \mathbb{C}$.
There is an action of $U(sl_2)$ on $SL_2^{\otimes k}$ that is the analogy of the diagonal action
by conjugacy of $SL_2\mathbb{C}$ on the coordinate ring of the cartesian product of $SL_2\mathbb{C}$ with itself $k$-times, $C[SL_2\mathbb{C}^k]=SL_2^{\otimes k}$, so that the ring
of invariants can be identified with the characters of $SL_2\mathbb{C}$ representations of
the free group on $k$ letters.
From the Bullock paper in Commentari, or if you wish the Przytycki-Sikora paper about the same time, you can identify this ring with Kauffman bracket skein algebra with
$A=-1$ of a cylinder over any orientable surface $F$ whose fundamental group is the free group
on $k$ letters. That means we can use the algebra of Jones-Wenzl idempotents to construct things.
Let $F$ be a closed oriented surface of genus $g$ and let $F'$ be the result of removing an open disk from $F$.
Let $\sum_c (-1)^c(c+1)s_c(\partial F')$ be series coming from coloring $\partial F'$ with the $c$th Jones Wenzl idempotent. We can define a linear functional on the characters ring of $F'$ by letting
$$YM(\alpha)=\lim_{N\rightarrow \infty} \mu^{\otimes 2g}(\sum_{c=0}^N\alpha(-1)^c(c+1)s_c(\partial F'))$$ The linear functional cannot see handleslides across the boundary of $F'$. That was how
we defined the Yang-Mills measure in the paper.
Lets see if we can see it more algebraically.
Complete $ SL_2^{\otimes 2g}$ so that
$$ \zeta=\sum_{c=0}^{\infty}(-1)^c(c+1)s_c(\partial F') $$
is
in the completion and annihilates handleslides.
Here is how to complete. The Yang-Mills measure defines a symmetric pairing
$$<\alpha,\beta>=YM(\alpha\beta)$$ We say a sequence $\alpha_n$ is Cauchy if for every
character $\beta$, the sequence of complex numbers $<\alpha_n,\beta>$ is Cauchy. We complete
by including the characters of the free group into the equivalence classes of Cauchy sequences.
The completion is no longer an algebra, but it is a module over the the characters of the free group.
Notice for any character $\alpha$, the sequence of partial sums of $\alpha \zeta$ is Cauchy,
and hence defines an element of the completion. The span of all $\alpha\zeta$, the cyclic module
generated by $\zeta$ is isomorphic to the $SL_2\mathbb{C}$ characters of the fundamental group
of $F$. Furthermore the restriction of the extension of the $\mu^{\otimes 2g}$ to that cyclic
module is the Yang-Mills measure.
Finally, instead of using characters, you can use the Kauffman bracket skein algebras of cylinders over surfaces as long as $A$ doesn't lie on the unit circle, or it it does, it needs to be a $2p$th root of unity for some counting number $p$.
There are other ways of getting there. I always thought that the Yang-Mills measure had something
to do with type $II_1$ factors, at least in the quantized case, but I could never get there. No doubt
you need $A$ to be real, and then you need to do some some sort of transform as initially skeins act like unbounded operators.
