[Collecting some of the comments in a community wiki answer.]

It's not clear whether you're asking about upper bounds or lower bounds.

At first glance, it seems you are asserting that in dimensions other than 8 and 24, there do not exist "very particular algebraic lattice structures" akin to $E_8$ and the Leech lattice, which achieve the optimal packing density. Depending on what you mean by "very particular algebraic lattice structures," this is probably not true; for example, it is widely believed that $E_6$ and $E_7$ achieve optimal packing density in their respective dimensions. There cannot be any evidence that $E_6$ and $E_7$ do not exist, because they do in fact exist.

Your intended question may really be about upper bounds. In dimensions 8 and 24, the packings are so good that, even before Viazovska's work, Cohn and Elkies were able to construct auxiliary functions based on linear programming that came close to proving optimality. Viazovska's breakthrough can be thought of as a new technique for constructing better auxiliary functions. It is natural to ask whether this technique could be extended to other dimensions, or whether there is some barrier.

If you look at the earlier work by Cohn and Elkies, you can see that in no other dimension > 3 was the gap between the lower and upper bounds as small as in dimensions 8 and 24. See Figure 1 in their paper, New upper bounds on sphere packings I. That might already suggest that replicating the "magic" in dimensions 8 and 24 in other dimensions is unlikely, but it doesn't say whether there is some intrinsic barrier to using linear programming techniques to close the gap.

The paper Dual linear programming bounds for sphere packing via modular forms by Cohn and Triantafillou takes a step toward answering this question by computing a dual linear programming bound, which establishes a limit to how good a (primal) linear programming bound can be. Their results apply when the dimension is a multiple of 4, and show that in some cases (most notably in dimensions 12 and 16, where the best lattice packings are widely believed to be optimal), there is indeed a provable barrier to using linear programming to prove optimality.

Of course, this still does not show that there could not be some other technique, perhaps based on semidefinite programming or some other entirely different idea, which establishes the optimality of $E_6$, $E_7$, or some of the other known "nice" lattice packings.

Finally, of course, the strongest "obstruction" to a claim that some "very particular algebraic lattice structure" achieves optimal packing density would be a non-lattice packing that achieves a higher density than any lattice packing. Not much is known rigorously in this direction, but see this MO question for some information.