My answer to the question:
http://http://mathoverflow.net/questions/104362/moduli-interpretations-for-modular-curves
http://mathoverflow.net/questions/104362/moduli-interpretations-for-modular-curves
led me to wonder about the question in the present title.
It seems that modular forms for non-congruence subgroups may not "arise" from algebraic geometry in the same way modular forms for congruence subgroups do (as is evidenced by the lack of a good moduli problem and this whole unbounded denominators thing). Nonetheless, they are clearly complex-analytic objects. I wonder if there's a $p$-adic analytic analog to be had.
Here are two more precise questions:
(less ambitious) Are there interesting congruences to be had between such modular forms? Interesting $p$-adic limits? Interesting $p$-adic families?
(more ambitious) Is there some (perhaps inherently analytic) description of non-congruence forms in terms of moduli of elliptic curves that can be mimicked in the $p$-adic analytic setting?
My rather vague feeling is that the Atkin and Swinnerton-Dyer congruences suggest that there's something to be said here, but I haven't been able to dig up much on these questions in particular. Does anyone know of such work?

