One way to interpret this question:
Should we be able to guess or morally justify the philosophy behind the Langlands conjecture without analyzing important examples and special cases?
I'm going to answer this one "no, we shouldn't". The Langlands conjectures emerged from Langlands' construction of the $L$-function of an automorphic form and his prior knowledge of Artin $L$-functions of a Galois representation, class field theory, and other topics. Wiping this away, I don't see how one could have guessed it.
It's true that both sides can be described as "arithmetic symmetry", sort of (maybe one would better be described as "symmetric arithmetic"), but lots of things can be described as "arithmetic symmetry" at this level of detail, and they're not all in correspondence. Some examples are automorphisms of arithmetic schemes, fundamental groups of arithmetic schemes, and actions of fundamental groups on things other than vector spaces.
It's true that we are more likely to see a correspondence between two classes of highly symmetric, arithmetic objects than between one such class and one class of non-symmetric or non-arithmetic objects. For example, we are unlikely to find a deep correspondence between Galois representations and finite graphs.
Thus I think your approach is somewhat backwards. The way to guess the Langlands correspondence is to first notice that special cases of the two sides correspond, and that concrete features appear similar on both sides, using easier steps like the Satake isomorphism, class field theory, and so on, and then generalize. Trying say "I believe a priori there should be some correspondence between these two types of objects, now I just have to figure out which ones match up with which ones and what that means in concrete terms" is trying to make the hard parts easy and the easy parts hard.
The closest thing I can imagine to this is to take the physical perspective on geometric Langlands, derive it from physical ideas (electric-magnetic duality in $N=4$ supersymmetric Yang-Mills), and deduce that it should have some arithmetic analogue or consequence. This faces two challenges, one is that making the translation is very hard (even having access to both the physical and arithmetic-geometric sides, it took Kapustin and Witten a lot of effort to see how they match up) and the physical ideas are not "morally obvious" in themselves.
This perspective gets the closest to seeing the two sides as "fundamentally the same". We think of representations of the Galois group into $\hat{G}$ as $\hat{G}$-bundles with flat connection, and we think of points of $G(F)\backslash G(\mathbb A_F)/ G( \hat{\mathcal O}_F)$ as bundles without flat connection (where the flat connection reappears when we take the cotangent bundle of the moduli space of bundles), and we compare the two types of bundles. But the comparison is not anything simple - note that $G$ is not $\hat{G}$ - it's roughly supposed to be an advanced type of Fourier transform.
One reason it's hard to imagine understanding this without getting into the weeds is that any philosophical description of how the Galois group of $\mathbb Q$ relates to representations of arithmetic groups over $\mathbb Q$ seems like it would apply equally well to how the Galois group of $\mathbb Q(x)$ relates to representations of arithmetic groups over $\mathbb Q(x)$. However, the first step of the Langlands program, class field theory, fails in that setting. There are notions of higher class field theory, but they don't just involve comparing representations and representations, and they don't seem likely to generalize to non-abelian groups. So we're saying something special about number fields that doesn't hold for higher-dimensional fields. (From my arithmetic-geometric perspective, I would say class field theory is a manifestation of Poincare duality, and the statement of Poincare duality depends heavily on the dimension. From the physical perspective, Langlands comes from a duality in four-dimensional field theory, and you don't get the same duality in higher dimensions.)
Why reductive groups and not general groups?
The rule of thumb you should always use in these types of things is that before conjecturing something for reductive groups you should test it for $\mathbb G_m$, and before conjecturing something for unipotent groups, or non-reductive groups, you should test it for $\mathbb G_a$. Characters of $\mathbb G_m(\mathbb A_F)/\mathbb G_m(F)$ correspond to characters of the Galois group by class field theory. Characters of $\mathbb G_a(\mathbb A_F)/\mathbb G_a(F)$ correspond to elements of $F$, not anything Galois-theoretic. What more is there to say?
