@David I would say a quotient of $\mathbb{Z}^n$, where $n$ is a natural number. Similarly, a finitely presented abelian group is a cokernel of a linear map $\mathbb{Z}^m \to \mathbb{Z}^n$. For those cokernels we constructively have the structure theorem (using the Smith normal form). Also this notion of finitely presented coincides with the general categorical notion of a compact object in a category.

@Mike Did so. But be encouraged to not accept the answer, as the true question is still open. I'm quite interested in a non-cheating answer myself.

@David: Thanks! I didn't know this condition before. The phrase "does not properly" probably contains a few too many negations to be constructively sensible, but it would be interesting to ponder a more positive reformulation. There is a different definition by Richman, specifically devised to work in a choiceless context: Don't consider ascending sequences, but ascending trees. With this definition a scheme $X$ is locally Noetherian if and only if $\mathcal{O}_X$ is Noetherian from the internal point of view of the topos $\mathrm{Sh}(X)$.

@Rachmaninoff: Yes. Check out On the spectrum of a ringed topos by Myles Tierney, published in Algebra, Topology, and Category Theory: A Collection of Papers in Honor of Samuel Eilenberg (edited by Alex Heller and Tierney). It's partially available on Google Books. I should add that the article is very nice and a joy to read.

Any subclass $\mathcal{C}$ of an abelian category determines a smallest Serre class containing it, by iteratively adding (the zero object and) the object $Y$ for any exact sequence $X \to Y \to Z$ where $X$ and $Z$ are objects of the previous stage. Note the missing zeros; alternatively, one can iteratively add the zero object, subobjects, quotients, and extensions. Anyway, this construction can in particular be performed with the class $\mathcal{C}$ of finitely generated abelian groups. Classically, its closure will coincide with $\mathcal{C}$, as $\mathcal{C}$ is already a Serre class.

It's nice that even a primary-school pupil, maybe armed with a calculator, can verify that $\mathbb{Z}_{10}$ is not an integral domain. Namely, such a child might wonder: Is there a number $x$ such that the last few digits of $x^2$ coincide with those of $x$? $5$ is an obvious first choice, but more digits are possible: $25$ ($25^2 = 625$), $625$ ($625^2=390625$), $90625$ ($90625^2 = 8212890625$), and so on. These approximations actually describe a single $10$-adic number $x$ which satisfies the identity $x^2=x$, or equivalently $x(x-1)=0$. Since $x$ is neither zero nor one, the claim follows.

I too would be interested in this document.

Over at the n-category café there was a slightly related question.

Here is one more class of examples (somewhat related): Let $X$ be a nonreduced scheme. Then the canonical morphism $X_{\mathrm{red}} \to X$ is topologically the identity, but of course not scheme-theoretically. ($X_{\mathrm{red}}$ is topologically $X$, but equipped with $\mathcal{O}_X/\sqrt{(0)}$ as structure sheaf.)

"you shouldn't think of radical ideals as points, but rather as a natural candidate for 'subsets of points'" I'd like to stress this sentiment. For any commutative ring (not necessarily reduced or finitely generated), the frame of open subsets of $\mathrm{Spec}A$ is canonically isomorphic to the frame of radical ideals of $A$. There is a notion of "point of a frame" (rather "point of the induced locale"). The points of this frame are in canonical bijection with the prime filters of $A$. If classical logic is available, then these in turn are in canonical bijection with the prime ideals of $A$.

@Urs: There is an internal characterization of the quasicoherence of a sheaf in the little Zariski topos of a scheme; see these notes. Briefly, a sheaf of modules $\mathcal{F}$ is quasicoherent if and only if, from the internal point of view, for any $f : \mathcal{O}_X$ the localized module $\mathcal{F}[f^{-1}]$ is a sheaf with respect to the modal operator $(\text{$f$ invertible} \Rightarrow \cdot)$.