Question
Let $G$ be a group, and let $X$ be a $G$-biset that is (weakly) invertible with respect to the contracted product. Is $X$ necessarily a bitorsor?
Background
By $G$-biset, I mean a set equipped with commuting left and right $G$-actions. There is a standard tensor product on the category of $G$-bisets called the contracted product; it is defined by $X \times_G Y = X \times Y / (x \cdot g, y) \sim (x, g \cdot y)$, where $G$ acts on the left by its left action on $X$, and on the right by its right action on $Y$. The unit object is the group $G$, where $G$ acts by left and right multiplication on itself.
A left $G$-torsor is a left $G$-set $X$ such that the map $G \times X \to X \times X$, $(g, x) \mapsto (g \cdot x, x)$ is a bijection. A right $G$-torsor is defined analogously. A $G$-bitorsor is a $G$-biset that is both a left and a right $G$-torsor. A $G$-bitorsor $X$ is necessarily invertible with respect to the contracted product; its inverse is the opposite $G$-bitorsor $X^{\operatorname{op}}$. This bitorsor has the same objects as $X$, but $g \in G$ acts on the left (resp. the right) by the right (resp. left) action of $g^{-1}$.
It follows from a simple counting argument that when $G$ is a finite group, any invertible $G$-biset is a $G$-bitorsor. Is this true for arbitrary groups (and more generally, in an arbitrary topos)? What about if we replace "invertible" by "right- (or left-) invertible"?
I can show, at least in the punctual topos (and I think it's true in general), that if $X^{\operatorname{op}}$ is an inverse to $X$, then $X$ must be a $G$-bitorsor. So the question is whether a $G$-biset can have an inverse not of this form.
The reason I'm interested in this question is that I want to understand how to generalize bitorsors to higher categorical settings. A possible generalization would be an invertible profunctor, but this is only a good definition if the answer to my question is affirmative.
