Appeals-to-authority, of course, are widely regarded in many contexts as logical flaws, as something to be avoided in logical discourse. But I believe that there is a sense, nevertheless, in which appeals-to-authority are a commonplace in mathematics. Namely, we all cite other mathematician's work in our own work, and such citations amount in a sense to an appeal to authority, appealing to the authority of that other mathematician and of the editors and referees of that journal that the cited theorem is correct, even though a proof of it is not immediately given in the citing text. Although it is sometimes stated that a mathematician should only cite work that he or she has personally checked in detail, this suggestion is surely not universally observed. In any case, even when it is observed, the citation itself still does not provide the full logical force of the conclusion that is drawn from it---one must still trust the cited theorem in order to proceed with the logical conclusion at hand. Thus, I find it to be essentially an appeal to authority.
But this kind of appeal to authority seems relatively mild in comparison with the truly objectionable kind one might imagine in other contexts. The reason is the open nature of the appeal; it can in principle be checked. That is, when we cite a published theorem, our readers can in principle look up the proof of that theorem themselves and determine if it is correct.
A more pernicious instance of appeal-to-authority, of course, would be less open, and not easily checkable. Although such appeals sometimes occur in mathematics, I believe that most mathematicians do not regard them as constituting proof. And so I don't think that we actually have a problem with this. Rather, when a mathematicians says that so-and-so famous mathematician asserts $X$, then I think that this by itself is rarely taken as a proof of $X$. Instead, it is taken as an indication that there likely is a proof of $X$, and that perhaps that famous mathematician has such a proof, and that perhaps we might search for such a proof ourselves.