I'm proposing the following statement. I'm relatively sure it is true. But I have to admit that a full proof might require a bit of work to fill all the gaps. What I mean is that if you really need to use it somewhere, I wouldn't quote this post as a proof !
Claim: A completely distributive boolean topos is the topos of $G$-set for $G$ a groupoid.
So $G$-set for $G$ a groupoid is the next best thing to slice of the category of sets: it is a category that is locally a slice of the category of sets. Also the result mentioned in the question can essentially be recovered (without too much work) as a special case of this one.
I' don't think I need to explain what is a Boolean Topos, nor why this is a reasonable candidate to repalce boolean algebras. The tricky notion is "completely distributive".
For a regular cardinal $\kappa$, $\kappa$-distributive Grothendieck topos or $\kappa$-topos, is a $\kappa$-exact localization of a presheaf category. Usual formal arguement should implies that this is equivalent to the assumption that the "colim" functor from the category of small presheaves of the topos $Psh(\mathcal{T}) \rightarrow \mathcal{T}$ commutes to $\kappa$-small limits.
Note that commutation of this functor to finite limits is a known way to encode all of the usual Giraud's axioms of a topos except accessibility, see for example Lack-Garner
By totally distributive I mean $\kappa$-distributive for all $\kappa$, which seems to also be the definition of the papers mentioned by Mike Shulman in the comments.
$\kappa$-distributivity can also be rephrased in a way that is a little closer to distributivity of lattices: C.Espindola has some papers on infinitary categorical logic where he studied a bit these $\kappa$-toposes. He has observed that this condition of $\kappa$-distributivity on a topos can be rephrased as, informally:
"a $\kappa$-small transfinite composition of covering sieve is a covering sieve".
this has to be formalized using trees. i.e. given a $S$ a tree where each branch has height less than $\kappa$, and $D:S^{op} \rightarrow \mathcal{T}$ a functor such that for each node of the tree $s$, the childrens of $s$ form a covering of D(s) (a jointly epimorphic familly).
For each branch $b$ of the tree, you can form its limit $S_b$, then his condition is that for each such diagram, the $S_b \rightarrow S_0$ (with $S_0$ the root) form a covering of $S_0$.
For a topos this is equivalent to $\kappa$-distributivity.
As we are assuming this for all $\kappa$ there might be further simplification on this condition (and one can maybe get rid of these trees in this case) But I'm not sure about it at this point.
Now:
Claim 2 A totally distributive Grothendieck topos is a presheaf topos.
This seems to follow from lemma 1 in the second paper linked by Mike, though it is late and I have some trouble following this paper that I discovered a few minutes ago only. The real reason I believe this claim is true is because of an unpublished result of C.Espindola in which I'm relatively confident and which implies this.
It is then a classical result (exercise ? ) that a presheaf topos $Prsh(I)$ is boolean if and only if $I$ is a groupoid. Hence the first claim follows from this second one.