What is the field with one element? I've heard of this many times, but I don't know anything about it.
What I do know is that it is supposed to solve the problem of the fact that the final object in the category of schemes is one-dimensional, namely $\mathop{\text{Spec}}\mathbb Z$.
So, what is the field with one element? And, what are typical geometric objects that descend to $\mathbb F_1$?
 A: Not an expert, but the idea is that there should be some "thing" over which we can define functors which, after base change, are schemes.  Over this "thing", rings of integers should be literal curves.  The big example of things that I know is that a vector space over F_1 (or F_un, if you're the sort who likes fun notation) is a pointed set.  For more info, there have been some "This Week's Finds" posts about them, and also this series at neverendingbooks.
A: As other have mentioned, F_1 does not exist of a field. Tits conjectured the existence of a "field of characteristic one" F_1 for which one would have the equality G(F_1) = W, where G is any Chevalley group scheme and W its corresponding Weyl group. 
Later on Manin suggested that the "absolute point" proposed in Deninger's program to prove the Riemann Hypothesis might be thought of as "Spec F_1", thus stating the problem of developing an algebraic geometry (and eventually a theory of motives) over it.
There are several non-equivalent approaches to F_1 geometry, but a common punchline might be "doing F_1 geometry is finding out the least possible amount of information about an object that still allows to speak about its geometrical properties". A "folkloric" introduction can be found in the paper by Cohn Projective geometry over F_1 and the Gaussian binomial coefficients.
It seems that all approaches so far contain a common intersection, consisting on toric varieties which are equivalent to schemes modeled after monoids. In the case of a toric variety, the "descent data" that gives you the F_1 geometry is the fan structure, that can be reinterpreted as a diagram of monoids (cf. some works by Kato). What else are F_1 varieties beyond toric is something that depends a lot on your approach, ranging from Kato-Deitmar (for which toric is all there is) to Durov and Haran's categorical constructions which contains very large families of examples. A somehow in-the-middle viewpoint is Soule's (and its refinement by Connes-Consani) which in the finite type case is not restricted only to toric varieties but to something slightly more general (varieties that can be chopped in pieces that are split tori). No approach is yet conclusive, so the definitions and families of examples are likely to change as the theory develops.
Last month Oliver Lorscheid and myself presented an state-of-the-art overview of most of the different approaches to F_1 geometry: Mapping F_1-land: An overview of geometries over the field with one element (sorry for the self promotion).
A: Charles mentioned "This Week's Finds" posts on the field with one element. There are a number of posts, but the following link is probably the best place to start.
A: Update: at the bottom there's a wonderful and fresh reference.

There's no field with one element in the literal sense, but there are constructions that work over different fields $\mathbb F_q$ and which sometimes make sense when $q=1$. Examples would include representation theory of $GL_n(\mathbb F_q)$ which, if I'm correct, becomes the representation theory of $S_n$ under that limit. 
In particular, indeed, vector spaces — the objects on which $GL_n(\mathbb F_q)$ act — should become sets, the objects on which $S_n$ acts.
Though I'm not an expert on $\mathbb F_1$, I've encountered the viewpoint you're referring to. It's not hard to see why people expect $\mathbb F_1$ to be the universal base: you usually expect that  $\mathop{\text{Spec}} \mathbb F_n \to \mathop{\text{Spec}} \mathbb F_m$ exists iff $m$ divides $n$, so $\mathop{\text{Spec}} \mathbb F_1$ should be terminal. Of course, things are not literally that simple, e.g. what $\mathop{\text{Spec}} \mathbb Z \times \mathop{\text{Spec}} \mathbb Z$ over $\mathop{\text{Spec}} \mathbb F_1$ is? 
Also, $\mathop{\text{Spec}} \mathbb Z$ should be thought of the object of dimension 3 (I think it has dualizing complex in degree 3, and primes are similar to 1-dimensional knots), so there's a pressing need for existence of some smaller scheme. That smaller scheme better "have real dimension 1" so we can claim that $\mathop{\text{Spec}} \mathbb Z$ is an algebraic curve over it. One then tries to compactify it.
When you get stuck geometrically, it helps to lose some information and go to zeta functions. There this approach leads to some direct conjectures. The problem with zeta functions is that everything is hard. Still, search for absolute zeta and you'll see some real things being discussed.
It's been called absolute motives in the works I read, I think by Manin, Kontsevich, and some IHES people. The search for kontsevich absolute motives brought me up the article that I found a while ago, math/0702206.
This is also somehow related to noncommutative geometry of Connes (endomotives), in fact I think $\mathbb F_1$ is one of his favorite ideas (also owes a lot to Soulé and other people of course) The two articles I found to be of special interest are 0806.2401 and 0809.2926.
Also, here's a blog post about $\mathbb F_1$ with link to the introductory paper. In fact there was a whole blog about $F_{un}$, which disappeared (articles, unfortunately,  also disappeared from my RSS reader).

Here's a paper by Connes and Consani that seems to explain most the topics mentioned above: 

 Schemes over F1 and zeta functions, 0903.2024 

A: Here's probably the simplest manifestation of the field-with-one-element phenomenon.  Define a projective $n$-space of order $q$ to be a collection of points, lines, planes, etc. satisfying the usual incidence relations with the additional condition that every line has $q+1$ points on it, every plane has $q^2+q+1$ points on it, and so forth.  For $q$ a prime power, all such spaces come from the usual definition of projective $n$-space $\Bbb P^n(\Bbb F_q)$ over a finite field.
But a projective $n$-space of order $1$ is precisely the Boolean algebra of subsets of a set with $n$ elements!
(This example is due to Henry Cohn, and it has the virtue that any theorems one wants to prove in this abstract setting don't depend on the value of $q$.)
A: There was a great series on this on neverending books.  Sadly, the spin off website, "F_un mathematics" seems to have disappeared from the web.
A: One of the many resources pointed out above linked to the unpublished preprint by Kapranov and Smirnov called Cohomology determinants and reciprocity laws: the number field case. It's posted page by page in jpgs, but it is definitely worth looking at. They work out the details of vector spaces over F_1^n in detail and also relate the classical power residue symbol to determinants of morphisms of these vector spaces.
