## Is there a way of canonically labelling permutation groups?

When working with large numbers of graphs, a canonical labelling routine is essential as, after the initial cost of canonically labelling each graph, it permits isomorphism checks to be replaced with identity checks.

Currently I am working with large numbers of transitive permutation groups; the degree is small (40ish) but the size of group can be large, and my programs are gumming up due to having to perform large numbers of - not exactly isomorphism - but conjugacy checks. (I need to check each pair of groups to see if they are conjugate in the symmetric group.)

This process would be considerably easier if there was a known way of finding a canonical conjugate of the incoming permutation groups...

-
the question of finding a canonical form of the centraliser ring of a permutation group is already non-trivial to me. I do not know anything better than trying all the orderings of orbitals, canonising each edge-coloured graph you get this way, and pick the maximal. (if the orbitals are "different" in some way, e.g. have different subdegrees, you might use it to restrict to, say, ordering the orbitals wrt subdegrees) The centraliser ring in a canonical form would give you a natural fingerprint for conjugacy. – Dima Pasechnik Mar 4 2012 at 4:46
I'd also look into what actually happens when you check the conjugacy with your software (GAP? Magma?). Maybe it's doing something dumb... – Dima Pasechnik Mar 4 2012 at 4:50
Dima, I'm using Magma, but I do not know of any way to determine (directly) whether it is doing anything dumb. I will try and do some experiments and see whether it is doing certain obvious things. – gordon-royle Mar 6 2012 at 8:02

Babai, Codenotti, and Qiao (ICALP 2012 from Babai's homepage, also in Codenotti's thesis) show how to test permutational isomorphism (=conjugacy - but this might be another good phrase to search with) of permutation groups in time that is polynomial in the order of the groups and and simply exponential ($c^n$ for fixed $c$) in the size of the permutation domain. From a theoretical perspective, I believe this is the best known.

In the transitive case, the runtime is still simply exponential in the degree but only linear in the order, and in this amount of time they can in fact list all of the permutational isomorphisms. Listing the permutational isomorphisms allows you to pick a canonical labelling by, e.g., lexicographic order. (And $|G|2^{40}$ is way better than the naive bound of $|G|40!$: $2^{40} \approx 10^{12}$, while $40! \approx 10^{48}$.)

Now theoretical bounds on algorithms may not tell you anything about efficiency in practice. (Depending on how large your transitive groups are, even the linear factor of $|G|$ might be too much for you.) It is at least reasonable to think that their algorithm is not only theoretically more efficient but also more efficient in practice than the naive algorithm(s). And of course one can always hope that it will work for your instances anyways, or perhaps with a few additional heuristics.

-
 Yes, conjugate permutation groups are often said to be "isomorphic as permutation groups". – Harry Altman Aug 16 at 19:44

This seems to be one of the biggest holes in the practical isomorphism arsenal. Various non-trivial possibilities come to mind, but their efficiency relies a lot on a more basic question. Define (and efficiently compute) a total order on labelled finite permutation groups. Thus, for any two permutation groups $G_1,G_2$ given by permutation generators, determine whether $G_1\lt G_2$, $G_1=G_2$ or $G_1\gt G_2$, where $G_1=G_2$ iff they are exactly the same (not just isomorphic) and the order obeys the transitivity law. This is possible in principle: compute the entire groups and compare them lexicographically. But naturally we don't want to compute the entire groups.

-