There's no field with one element in the literal sense, but there are constructions that work over different fields `F_q` and which are sometimes make sense when `q=1`. Examples would include representation theory of `GL_n(F_q)` which, if I'm correct, becomes the representation theory of `S_n` when `q = 1`. 


In particular, indeed, vector spaces — the objects on which `GL_n(F_q)` act — should become sets, the objects on which `S_n` acts.

There's more to say about `F_1` from the point of view of absolute motives, but I'm not an expert. This is also related to Connes, in fact I think `F_1` is one of his favorite ideas.

Also, here's a whole blog about `F_1`: