Since I commented above on something quite general, here is an attempt at a specific contribution. It's not at all personal in that I'm referring to a well-known point of view in Diophantine geometry, whereby solutions to equations are sections of fiber bundles. Some kind of a picture of the fiber bundle in question was popularized by Mumford in his Red Book. I've discovered a reproduction on this page. The picture there is of $Spec(\mathbb{Z}[x])$$\operatorname{Spec}(\mathbb{Z}[x])$, but interesting equations even in two variables will conjure up a more complicated image of an arithmetic surface fibered over the 'arithmetic curve' $Spec(\mathbb{Z})$$\operatorname{Spec}(\mathbb{Z})$. A solution to the equation will then be a section of the bundle cutting across the fibers, also in a complicated manner. Much interesting work in number theory is concerned with how the sections meet the singular fibers.
Over the years, I've had many different thoughts about this perspective. For me personally, it was truly decisive, in that I hadn't been very interested in number theory until I realized, almost with a shock, that the study of solutions to equations had been 'reduced' to the study of maps between spaces of a quite rigid sort. In recent years, I think I've also reconciled myself with the more classical view, whereby numbers are some kinds of algebraic gadgets. That is, thinking about matters purely algebraically does seem to provide certain flexible modes that can be obscured by the insistence on geometry. I've also discovered that there is indeed a good deal of variation in how compelling the inner picture of a fiber bundle can be, even among seasoned experts in arithmetic geometry. Nevertheless, it's clear that the geometric approach is important, and informs a good deal of important mathematics. For example, there is an elementary but key step in Faltings' proof of the Mordell conjecture referred to as the 'Kodaira-Parshin trick,' whereby you (essentially) get a compact curve $X$ of genus at least two to parametrize a smooth family of curves $$Y\rightarrow X.$$ Then, whenever you have a rational point $$P:Spec(\mathbb{Q})\rightarrow X$$$$P:\operatorname{Spec}(\mathbb{Q})\rightarrow X$$ of $X$, you can look at the fiber $Y_P$ of $Y$ above $P$, which is itself a curve. The argument is that if you have too many points $P$, you get too many good curves over $\mathbb{Q}$. What is good about them? Well, they all spread out to arithmetic surfaces over the spectrum of $\mathbb{Z}$ that are singular only over a fixed set of places. This part can be made obvious by spreading out both $Y$, $X$, and the map between them over the integers as well, right at the outset. If you don't have that picture in mind, the goodness of the $Y_P$ is not at all easy to explain.
Many people from outside the area seem to have difficulty understanding the picture I mentioned because they are intuitively suspicious of its usefulness. Consider a simpler picture of the real algebraic curve that comes up when one studies cubic equations like $$E: y^2=x^3-2.$$ There, people are easily convinced that geometry is helpful, especially when I draw the tangent line at the point $P=(3,5)$ to produce another rational point. What is the key difference from the other picture of an arithmetic surface and sections? My feeling is it has mainly to do with the suggestion that the point itself has a complicated geometry encapsulated by the arrow $$P:Spec(\Bbb{Z})\rightarrow E.$$$$P:\operatorname{Spec}(\Bbb{Z})\rightarrow E.$$ That is, spaces like $Spec(\Bbb{Q})$$\operatorname{Spec}(\Bbb{Q})$ and $Spec(\Bbb{Z})$$\operatorname{Spec}(\Bbb{Z})$ are problematic and, after all, are quite radical.
In $Spec(\Bbb{Q})$$\operatorname{Spec}(\Bbb{Q})$, one encounters the absurdity that the space $Spec(\Bbb{Q})$$\operatorname{Spec}(\Bbb{Q})$ itself is just a point. So one has to go into the whole issue that the point is equipped with a ring of functions, which happens to be $\Bbb{Q}$, and so on. At this point, people's eyes frequently glaze over, but not, I think, because this concept is too difficult or because it competes with some other view. Rather, the typical mathematician will be unable to see the point of looking at these commonplace things in this way. The temptation arises to resort to persuasion by authority then (such and such great theorem uses this language and viewpoint, etc.), but it's obviously better if the audience can really appreciate the ideas through some first-hand experience, even of a simple sort. I do have an array of examples that might help in this regard, provided someone is kind enough to be still interested. But how helpful they really are, I'm quite unsure.