Commutative rings : Topoi = Fields :? The following is probably a bad question, but hopefully, it might have a  very good answer.
In category theory there is a quite famous analogy between topoi and commutative rings, I was never convinced by this analogy, but the best way to see how far an analogy can be pushed is to challenge it. Clicking on the link that I provided above you can have an extensive presentation of the analogy, the general motto can be grasped by the following table.

Remark 6.1.1.3. $\space$ Let $\mathcal{X}$ be an $\infty$-category. The assumption that colimits in $\mathcal{X}$ are universal can be viewed as a kind of distributive law. We have the following table of vague analogies:
$$\begin{array}{ccc} && \text{Higher Category Theory} && \quad && \text{Algebra} && \\ \hline \\ & & \infty\text{-Category} &  &  & & \text{Set} \\ \\ & & \text{Presentable } \infty\text{-category} & & & & \text{Abelian group} \\ \\ & & \text{Colimits} & & & & \text{Sums} \\ \\ & & \text{Limits} & & & & \text{Products} \\ \\ & & \varinjlim(X_\alpha) \times_S T \simeq \varinjlim(X_\alpha \times_S T) & &  & & (x + y)z = xz + yz \\ \\ & & \infty\text{-Topos} & & & & \text{Commutative ring} \end{array}$$
Definition 6.1.1.2 has a reformulation in the language of classifying functors ($\S$3.3.2):

That corresponds to Rem 6.1.1.3 in my version of HTT by Lurie.

Q. According to this analogy, what should be a field?

Maybe I should say why this might be a stupid question or even a stupid challenge for the analogy. In fact it might be the case that:

*

*The notion of field is interesting only in low dimension.

*The correct generalization of the notion of field looks very different in categories and trivializes for sets because of their intrinsic rigidity.

 A: This is a long comment. I would prefer to say that (Grothendieck) topoi are "(some) affine schemes over $\text{Spec } \text{Set}$." Here is my preferred version of the table, sprinkle $\infty$s according to taste: 


*

*Categories and functors : sets

*Presentable categories and left adjoints : abelian groups

*Monoidal presentable categories and monoidal left adjoints : rings

*Symmetric monoidal presentable categories and symmetric monoidal left adjoints : commutative rings


(Here all monoidal structures distribute over colimits, which I guess is equivalent to requiring that they be closed.) 
This more general setup allows for "2-affine algebraic geometry"; for example, $\text{QCoh}(X)$ for $X$ a scheme (stack, derived stack, etc.) is now an example, and in some nice cases covered by Tannakian theorems this embedding of algebraic geometry into "2-ring theory" is even fully faithful. 
We get closer to (Grothendieck) topoi by upgrading "symmetric monoidal" to "cartesian monoidal." If we upgrade "cartesian monoidal" to "has all finite limits" (and upgrading the functors to being left exact) we get almost all the way towards logoi (topoi and algebraic morphisms), which in this analogy are "(some) commutative $\text{Set}$-algebras." Topoi and geometric morphisms are the geometric objects corresponding to these commutative ring-like objects. 
Example. Let $G$ be a discrete group and consider the logos $\text{Set}^G$ of $G$-sets. Algebraic morphisms from $\text{Set}^G$ to a logos $L$ correspond to $G$-torsors in $L$ (by Diaconescu's theorem), and accordingly "$\text{Spec } \text{Set}^G$" ($\text{Set}^G$ regarded as a topos) is a "2-affine" version of the stack $BG$. 

It's not clear to me whether there's a compelling generalization of field here. One definition of a field is that it's a commutative ring with no nontrivial quotients (effective epimorphisms out); I don't know enough about topos theory to know if logoi have a useful notion of quotient or epimorphism. 
A necessary condition might be "has at most one point," where here a point is a geometric morphism from / algebraic morphism to $\text{Set}$. This includes $\text{Set}$ (an avatar of $\mathbb{F}_1$?) and $\text{Set}^G$ for $G$ a group but excludes, for example, $\text{Sh}(X)$ for $X$ a topological space with at least two points. 
I like Simon Henry's proposal that $\text{Set}$ is the only field. This would mean that $\text{Set}$ has no nontrivial "field extensions." It certainly seems to have no nontrivial "Galois extensions." 
A: I've played a little bit with this question in the past, and I don't have anything firm, but here are some thoughts:


*

*It seems to me that the characteristic properties of fields have a lot to do with the Zariski topology. For instance, fields can be characterized by the fact that they have no nontrivial localizations (i.e. Zariski-open subsets) or by the fact that they have no nontrivial quotients (i.e. Zariski-closed subsets).

*This poses a bit of a challenge, because the easiest topology to see an analog of on the topos side of the analogy is not the Zariski topology, but rather the etale topology. For instance, an etale geometric morphism is a pretty natural thing to consider, and (I think?) a pretty good analog of an etale map in algebraic geometry.
Nevertheless, we can still think about what an analog of the Zariski topology on toposes might be. 


*

*The most straightfoward thing to do would be to take the analog of a Zariski-open set in $X$ to be an open embedding $U \to X$. I believe this is the same thing as an open subtopos, corresponding to a subobject of the terminal object, i.e. a point of the subobject classifier. So we might define a field to simply be a topos such that the subobject classifier has just two points -- a two-valued topos.


But here is where I think it's fruitful to think in terms of $\infty$-toposes:


*

*Mike Shulman has argued (see the first bullet point at the link) that open geometric morphisms are just the 0th step in a hierarchy of locally $n$-connected geometric morphisms, and at the top we find locally $\infty$-connected geometric morphisms. Similarly, various other "named" types of geometric morphisms are best seen as lying in a hierarchy of types of $\infty$-geometric morphisms.

*Most relevantly here, there's a natural hiearchy which interpolates between "open subtopos" and "etale topos". Namely, for each $n$, we can consider toposes of the form $\mathcal T / X$ where $X$ is $n$-truncated; etale corresponds to the case $n = \infty$ while open corresponds to the case $n = -1$. So for each $n$, there's a corresponding notion of "$n$-field" -- a topos $X$ with no nontrivial $n$-truncated objects (where "$\infty$-field" is analogous to an algebraically closed field, but turns out to just mean the trivial topos).
Somehow, that doesn't sound super-useful to me. Probably I've messed up some part of the analogy.
