Grothendieck was aware (at least) of modular forms and their relations to motives by J P Serre in a letter dated December 31, 1986.

[![enter image description here][1]][1]


  [1]: https://i.sstatic.net/YgaPe.jpg

Grothendieck (also) develop on "La longue Marche" a theory of "topos modulaire" (now completely available online!).