Are there algebraic structures that integrate groups with probability measures? For instance, can the closure operation on a group be assigned a probability that says "how much" a member belongs to the group, kind of generalizing the rigid definition of making a member belong to the group. It will be great if experts here can point to such structures and recommend some books. Thanks.

3

hypergroupmight be in the right spirit. You can think of a hypergroup as a weaker version of a group, in the sense that given two point masses $\delta_x$ and $\delta_y$ we no longer have a single "location" for the product (i.e. $\delta_{xy})$ but instead the "product" of $\delta_x$ and $\delta_y$ is a probability measure. $\endgroup$