There's no field with one element in the literal sense, but there are constructions that work over different fields `F_q` and which sometimes make sense when `q=1`. Examples would include representation theory of `GL_n(F_q)` which, if I'm correct, becomes the representation theory of `S_n` under that limit. 


In particular, indeed, vector spaces — the objects on which `GL_n(F_q)` act — should become sets, the objects on which `S_n` acts.

Though I'm not an expert on `F_1`, I've encountered the viewpoint you're referring to. It's not hard to see why people expect `F_1` to be the universal base: you usually expect that  `Spec F_n -> Spec F_m` exists iff `m` divides `n`, so `Spec F_1` should be terminal. Of course, things are not literally that simple, e.g. what `Spec ZZ x Spec ZZ` over `Spec F_1` is? 

Also, `Spec ZZ` should be thought of the object of dimension 3 (I think it has dualizing complex in degree 3, and primes are similar to 1-dimensional knots), so there's a pressing need for existence of some smaller scheme.
 
When you get stuck geometrically, it helps to lose some information and go to zeta functions. There this approach leads to some direct conjectures. The problem with zeta functions is that everything is hard. Still, search for `absolute zeta` and you'll see some real things being discussed.

It's been called `absolute motives` in the works I read, I think by Manin, Kontsevich, and some IHES people. The search for `kontsevich absolute motives` brought me up the article that I found a while ago, **[math/0702206][1]**

This is also somehow related to noncommutative geometry of Connes (`endomotives`), in fact I think `F_1` is one of his favorite ideas (or is it from some other part of French math circle?) 

Also, here's a [blog post about `F_1`][2] with link to the introductory paper. In fact there was a whole blog about `F_un`, which strangely disappeared (well, at least the articles I wanted to read should be still somewhere cached at my other computer).


  [1]: http://arxiv.org/abs/math/0702206
  [2]: http://noncommutativegeometry.blogspot.com/2008/06/fun-day-two.html