If $I$ is a set, $U$ a nonprincipal ultrafilter on $I$ and $E=(E_i)_{i\in I}$ a family of sets indexed by $I$, then the ultraproduct $E^*$ of $E$ is generally defined as the quotient of $\prod_{i\in I}E_i$ by the equivalence relation "equality on a subset of $I$ which belongs to $U$".

However, this definition is "wrong": this $E^*$ is nonempty if and only if *all* the $E_i$'s are nonempty, while the expected condition is "if and only if $\{i\in I\vert E_i\neq\emptyset\}\in U$". In fact, Łoś' theorem is false with this definition since "nonempty" can be defined by the formula $(\exists x)(x=x)$.

So I guess the right definition is $E^*=\varinjlim_{J\in U}\prod_{i\in J}E_i$ where $U$ is ordered by reverse inclusion and the transition maps are the projections. If each $E_i$ is nonempty this is equivalent to the standard form, which explains why the latter is used since in most applications (at least in algebra) the $E_i$'s carry some algebraic structure which excludes emptiness.

Of course there is a sheaf-theoretic version of this: let $I^\vee$ be the Stone-Čech compactification of the discrete space $I$, $j:I\to I^\vee$ the canonical inclusion. Then $(E_i)$ defines a sheaf of sets $\mathcal{E}$ on $I$. Put $\mathcal{E}^\vee:=j_*\mathcal{E}$. Then $E^*$ is the stalk of $\mathcal{E}^\vee$ at hte point corresponding to $U$. (In the case of ultraproducts of rings, this is explicitly stated in Schoutens' LNM 1999 book: see 2.6.2 there).

My question: are there accessible references where this issue is correctly addressed?