The most elementary way to define $p$-adic modular forms is via limits of classical modular forms. More precisely $f \in \mathbb{Z}_p[[q]]$ is called a $p$-adic modular form if there are modular forms $f_n$ with integral coefficients such that $f \equiv f_n \mod p^n$ (as $q$-expansions). Note it does not really make sense to attribute 'a weight' to $f$ since the $f_n$ are allowed to have different (increasing weights). This is the older definition by Serre.

I know (but I do not understand) a newer definition by Katz, which has a more geometric flavor. See here math.arizona.edu/~swc/notes/files/01BuzzardL2.pdf.

So we have an approach using the $q$-expansion and we have an approach to $p$-adic modular forms using geometric ideas.

My question now is, whether there is also a developed theory on analytic p-adic modular forms?

Some ideas what this might mean. For example we could consider the Eisenstein series $$ E_4(\tau)=\sum_{n,m \in \mathbb{Z}} \frac{1}{(n\tau+m)^4} $$ as function of $\tau$ not being an element of the upper half-plane but of some subset of $\mathbb{C}_p$.

Does this sum even converge somewhere in $\mathbb{C}_p$. And is it (up to a constant) a classical or Katz $p$-adic modular form? Does it even equal (mod p) the Eisenstein series $E_4$?

A similar question could be posed for the infinite product usually defining the $\Delta$-function $$ q\prod_{n=1}^\infty (1-q^n)^{24}. $$ Does this converge somewhere when $q$ is in some subset of $\mathbb{C}_p$. Is it a $p$-adic modular form?

If there is no such theory? Why not? Is it not interesting?

reallya function on the upper half plane! It's a function on elliptic curves, or pairs consisting of an elliptic curve and a differential. This definition works well $p$-adically and over the complexes. Now over the complexes every elliptic curve is isomorphic to $\mathbf{C}/\langle 1,\tau\rangle$ for some $\tau$ in the upper half plane. But over the $p$-adics this is not true; the curves that can be uniformised are those with bad reduction. So for a classical modular form... – Kevin Buzzard May 24 '11 at 20:04