There's no field with one element in the literal sense, but there are constructions that work over different fields F_q and which are sometimes make sense when q=1. Examples would include representation theory of GL_n(F_q) which, if I'm correct, becomes the representation theory of S_n when q = 1.
In particular, indeed, vector spaces — the objects on which GL_n(F_q) act — should become sets, the objects on which S_n acts.
There's more to say about F_1 from the point of view of absolute motives, but I'm not an expert. This is also related to Connes, in fact I think F_1 is one of his favorite ideas.
Also, here's a whole blog about F_1: