First, if the length is odd, say, $F:=ABABA=BABAB$ then $ABABABABAB=F^2=BABABABABA$. So if the braid relation is satisfied for some odd length then it is satisfied for some even length. Therefore we can solve for even lengths only.

Now let's say there exists a natural number $m$ such that $C:=(AB)^m=(BA)^m$. We have that if $C:=ABABAB=BABABA$ then
$$AABABAB=ABABABA$$
$$AABABAB=BABABAA$$
$$AC=CA$$

Similarly,
$$BABABAB=BBABABA$$
$$ABABABB=BBABABA$$
$$CB=BC$$

By this, if $C$ has $n$ distinct eigenvalues then $AB=BA$. Check if $BA=AB$. This is a sufficient condition. It is also necessary if $C$ has $n$ distinct eigenvalues. How can we check that? Look at the eigenvalues of $AB$ (which are the same as the eigenvalues of $BA$). Since $A$ and $B$ are unitary, $AB$ is unitary. So its eigenvalues are on the unit circle. They are equivalent if there exists a natural number $m$ such that $\mathrm{e}^{im\theta}=\mathrm{e}^{im\phi}$. This happens iff there exists an integer $k$ such that $m(\theta-\phi)=2\pi k \iff \theta-\phi=q\pi$ for some rational number $q$. If $AB$ has no equivalent eigenvalues then $(AB)^m$ has distinct eigenvalues (no matter what $m$). In this case, a necessary (and sufficient) condition would be that $AB=BA$ which is easy to check.