My answer to the question:

moduli interpretations for modular curves

led me to wonder about the question in the present title.

It seems that modular forms for non-congruence subgroups may not "arise" from algebraic geometry in the same way modular forms for congruence subgroups do (as is evidenced by the lack of a good moduli problem and this whole unbounded denominators thing). Nonetheless, they are clearly complex-analytic objects. I wonder if there's a $p$-adic analytic analog to be had.

Here are two more precise questions:

(less ambitious) Are there interesting congruences to be had between such modular forms? Interesting $p$-adic limits? Interesting $p$-adic families?

(more ambitious) Is there some (perhaps inherently analytic) description of non-congruence forms in terms of moduli of elliptic curves that can be mimicked in the $p$-adic analytic setting?

My rather vague feeling is that the Atkin and Swinnerton-Dyer congruences suggest that there's something to be said here, but I haven't been able to dig up much on these questions in particular. Does anyone know of such work?

4more comments