Is there an notion of elliptic curve over the field with one element? As I learned from a previous question, there are several different versions of what the field with one element and what schemes over it should be (see for example this article by Javier López Peña and Oliver Lorscheid). What I want to know is whether there a good notion of elliptic curve over $F_{un}$? What about modular forms?

In a strict sense, elliptic curves over the rationals (say) are not defined over $F_1$ since their reduction modulo $p$ varies with $p$, e.g. they have places of bad reduction. However, CM elliptic curves have some of the properties that one would associate with objects defined over $F_1$. For example, their Lfunction looks a bit like the twist of a constant elliptic curve over a function field, except that the role played by the character is actually played by a Hecke character. So, in a sense, a CM elliptic curve is the twist of a curve over $F_1$ by a Hecke character, although I have never tried to fully formalize this notion. Another way to view this is perhaps through J. Borger's viewpoint in which a variety over $F_1$ is a variety in which all Frobeniuses lift. That sort of happens for CM elliptic curves. 


As was mentioned by others, currently varieties over $\mathbb F_1$ look uncomfortably like toric varieties or something very close to that. But of course there is a way to think of an elliptic curve as a sort of "toric" variety, using the Tate's uniformization $E\ "="\ \mathbb G_m/\mathbb Z\ $! (Same for higherdimensional abelian varieites, using the MumfordTate(FaltingsChai) uniformization.) So there has to be a way... Put it another way, an elliptic curve is very similar to $\mathbb P^1$ with two points, $0$ and $\infty$, identified. Now that is an object that is really defined over $\mathbb F_1$. 


The only places I am aware of where elliptic curves and F_1 have a common appearance are the articles link text (arXiv version2, not the newest) and link text, both by Connes and Consani and both roughly saying the same. It's not really about elliptic curves over Fun but they rather apply their Funpoint counting techniques to elliptic curves. The word modular form also occurs. My feeling is that elliptic curves should rather not belong to the things defined over F_un. If they don't in Connes/Consani's setting then they don't anywhere, since theirs is currently the most comprehensive notion of F_1scheme (I am not sure, though, about Soule's and Connes/Consani's first approach, which are hard to compare with the others) 


There is a good notion (I think) from Connes and Consani: every one of their $\mathbb{F}_1$schemes has a base change to $\mathbb{Z}$, which is an actual scheme. I don't think any of them have been constructed, and I believe some of the earlier notions ruled them out (for example, one of them only allowed toric varieties). Connes and Consani seem hopeful about their existence (the links in Peter's answer lead to them arguing heuristically about why their zeta functions look good), but I didn't see any sign of them having been constructed yet. 


In most of the current schemes, it is very unlikely that elliptic curves are defined over F_1. They are certainly not in Deitmar's or ToenVaquie since they restric to toric varieties. For Soule/ConnesConsani old notions, all the examples found so far seem to come from torified varieties (as defined by Oliver and myself in this paper. Also in CC new notion, up to the torsion part of the monoidal scheme their schemes appear to be generalized torified (see Theorem 2.2 in the reference you mention). But all torified schemes are rational, so elliptic curves are not in there. On the other hand, Manin was very confident that the set of torsion points of an elliptic curve would define a convenient model for it. But if that was the case I don't think it would fit CC models, but rather Manin/Marcolli analytic approach, conjecturally related to Borger's, but no there is clear explanation on that relation just yet. 

