Unfortunately the question I am asking isnt very welldefined. But I will try to make it as precise as possible. Supposed I am given a modp representation of $G_Q$ into $Gl_2(F_p)$. I want to check for arithmetic invariants so that I can conclude that the representation comes from a modular form but not an elliptic curve. The whole point of this exercise is to understand the difference between the representations coming from elliptic curves and cusp forms in general. I hope I was able to make the question precise. A few things that one can look at is the conductor of an elliptic curve (i.e. the exponent of 2 in the level of modular form is too high then it cant come from an elliptic curve) or one can look at the Hasse bound for $a_l$ for different primes. But I want to know some nontrivial arithmetic constraints attached to such invariants. Also if such a representation doesnt come from an elliptic curve then it must come from an abelian variety of $GL_2$ type. Can anything be said about that abelian variety in general.
1 Answer
Since your representation $\overline{\rho}$ is defined over $\mathbb F_p$, you can't do things like the Hasse bounds, since the traces $a_{\ell}$ of Frobenius elements at unramified primes are just integers mod $p$, and so don't have a welldefined absolute value.
One thing you can do is check the determinant; this should be the mod $p$ cyclotomic character if $\overline{\rho}$ is to come from an elliptic curve. In general (or more precisely, if $p$ is at least 7), that condition is not sufficient (although it is sufficient if $p = 2,3$ or 5); see the various results discussed in this paper of Frank Calegari, for example. In particular, the proof of Theorem 3.3 in that paper should give you a feel for what can happen in the mod $p$ Galois representation attached to weight 2 modular forms that are not defined over $\mathbb Q$, while the proof of Theorem 3.4 should give you a sense of the ramification constraints on a mod $p$ representation imposed by coming from an elliptic curve.

$\begingroup$ Ok I should have added that. I am assuming that the Galois representation is coming from a modular form so the determinant already has cyclotomic character. As for the example for p=7 there is indeed a form of level 29 and weight 2 whose mod 7 Galois representation doesnt come from a modular form. So that got me thinking what went wrong for that prime. As you have pointed out the condition is not sufficient for higher primes, that raises the natural question about the arithmetic of these representations. Anyway thanks a lot for your answer. I will look into Calegari's paper $\endgroup$– ArijitJun 25, 2010 at 3:55

2$\begingroup$ By "coming from a modular form", do you mean "modular form of weight 2 and trivial nebentypus"? In general, the Galois rep. coming from a modular form of weight k and nebentypus epsilon has determinant cyclotomic^(k1) epsilon (or the inverse of this, depending on conventions). Also, "doesn't come from a modular form" should probably read "doesn't come from an elliptic curve". $\endgroup$– EmertonJun 25, 2010 at 4:08

$\begingroup$ Oh I am being really very careless. I briefly looked at the paper. But it doesnt say anything about the abelian variety that corresponds to the modular form. I believe that it is not a very appropriate question because my knowledge in this field is really very limited. Thanks again Prof. Emerton for clarifying my doubts. $\endgroup$– ArijitJun 25, 2010 at 4:36