One question is whether Eichler really did say it. One of the nice things about Google searches these days is that they let you trace the etymology of words, phrases, and aphorisms. I see a chain of references to this quote associated with Andrew Wiles and Fermat's Last Theorem. The chain seems to begin with a BBC interview in 1996 or 1997 in which Wiles says, "There's a saying attributed to Eichler that there are five fundamental operations of arithmetic: addition, subtraction, multiplication, division, and modular forms." On the other hand, this recent paper by Edixhoven, van der Geer, and Moonen calls the quote apocryphal.

I can't trace what happened before that interview. Since the documented history of this quip currently begins there, we can start with what Wiles himself meant. The interviewer wanted to know what a modular form is, but that's not easy to explain on television. Obviously it was a facetious (and witty) answer. It was also part of a general point in the interview that modular forms are widely studied and seem fundamental.