There's no field with one element in the literal sense, but there are constructions that work over different fields `F_q` and which sometimes make sense when `q=1`. Examples would include representation theory of `GL_n(F_q)` which, if I'm correct, becomes the representation theory of `S_n` under that limit. 


In particular, indeed, vector spaces — the objects on which `GL_n(F_q)` act — should become sets, the objects on which `S_n` acts.

There's more to say about `F_1` from the point of view of absolute motives, but I'm not an expert (one think that people talk about is the mental picture of absolutely every scheme being kind of a scheme defined over `F_1`, search for `kontsevich absolute motives`). This is also related to Connes, in fact I think `F_1` is one of his favorite ideas (or is it from some other part of French math circle?) 

Also, here's a [blog post about `F_1`][1] with link to the introductory paper. In fact there was a whole blog about `F_un`, which strangely disappeared (well, at least the articles I wanted to read should be still somewhere cached at my other computer).


  [1]: http://noncommutativegeometry.blogspot.com/2008/06/fun-day-two.html