I doubt that a mathematically rigorous explanation of the phenomenon discovered in that paper exists using today's technology. While mathematical statistical mechanics is a well-developed field of study, there is still much to be done before the applications of statistical mechanics by physicists can be made rigorous.

Any explanation should include a mathematical description of the "before-shaking" and "after-shaking" configurations; this should include a rigorous definition as well as a proof of some of their basic properties.

The "after-shaking" configuration seems to be related to the usual close-packing of cubes, where the cubes are congruent to a configuration where their centers are on $\mathbb{Z}^3$. (Understanding how this relates to what is seen with the cylindrical boundary conditions seems hard though). The "before-shaking" configuration then seems to correspond to a "disordered jammed" packing of the cubes. So far as I know, the set of such packings is not understood mathematically (though I would be delighted to be proven wrong!).

Let me explain this a bit more. Let $\mathcal{C}_{N,L}$ be the configuration space of $N$ hard cubes in a periodic box of size $L\times L\times L$; this is a semialgebraic subset of the $6N$-dimensional space of all possible positions and orientations. One can discuss jamming of configurations in terms of the connected components of the family of these spaces as $L$ changes (see e.g. this old mathoverflow question on the configuration space of hard disks).

For $N$ small enough, presumably one can show that there are often jammed configurations of cubes which do not come from the close-packed cubic lattice state. However, we are really interested in the thermodynamic limit where $N,L\rightarrow\infty$ with the density $N/L^3$ fixed and so far as I know, no one has been able to say anything about the ensemble of these jammed configurations in that limit; for example, it would be nice to have a proof that there exists a disordered thermodynamic phase of jammed configurations.

Even in the absence of proof, we still have some idea of what happens as one changes the density (an understanding at the level of physics, not mathematics). See for example my old answer here for references to experimental and simulation work on packed cubes and some more discussion about jamming.

Anyways, here's the punchline:

We've run into all of these difficulties (even in the world of equilibrium statistical mechanics) and we haven't even discussed the real meat of the paper: the *driven, dissipative dynamics* of these hard cubes in the presence of gravity and curved boundary conditions and friction!

P.S. I doubt this was your intention, but saying without any qualification that an experimental paper "is not very rigorous" would ordinarily be interpreted as saying that the design of the experiments or the way they were carried out was sloppy. That is not my understanding from reading the paper.