Great question! Yes, this is missing from the original paper and is certainly needed for the main result. It is true though. There's two different proofs currently in the literature, Deshpande and Mukhopadhyay Corollary 2.11 and Davydov and Nikshych proof of Theorem 8.5.
You can also prove it using an argument I learned from Theo Johnson-Freyd, namely given an object $X$ in $C_g$ you can realize $C_g$ as $A$-mod in $C_e$ with $X$ corresponding to $A$ itself, then $C_{g^{-1}}$ can be identified with mod-$A$ and the dual object will be $A$ again and the evaluation and coevaluation maps are given by multiplication and the unit of the algebra structure on $A$.
As to your last two sentences, rigidity is pretty essentially to the whole setup, otherwise the graded parts don't need to be invertible and then you leave the world of homotopy theory. For example, take the semisimple category with two objects 1 and X with tensor product defined by $X \otimes X = 0$ (aka the universal counterexample to all questions involving rigidity), this is $\mathbb{Z}/2\mathbb{Z}$ graded but the odd part is not invertible as a bimodule over the even part.