I tried to digest Joel's and Mike's answers and came up with the following picture:
Starting with a model of pure and well-founded set theory we can see it as a class of dots with $\in$-arrows between them. In this picture, all sets have structural properties only, no intrinsical. Furthermore, all sets are distinguishable by their structural properties (Corollary 1.1 and Proposition 1.2 in Aczel's Non-Well-Founded Sets, p.5, if I understood this right). Especially there is exactly one empty set, exactly one singleton set containing the empty set, and so on.
Introducing atoms, we have the situtation, that the atoms cannot be distinguished by their structural properties: they all have no elements, they all are contained in exactly one singleton set, and so on.
But suddenly, also non-empty sets become undistinguishable: take any pure set and replace all its "recursive occurrences" of the empty set by one and the same atom, and you won't be able to distinguish the two structurally.
So atoms are no more structurally undistinguishable than normal sets (in the presence of atoms) - just like Joel says in one of his comments on his answer.
Together, these findings suggest that atoms may have no practical use in a structural set theory. Summarizing:
Atoms don't have intrinisical properties, but they have structural properties, just as any other set. They are not as propertyless as one might have wanted.
Furthermore, even normal sets may be seen as dots, having no intrinsical, but only structural properties.
In the presence of atoms, atoms are no more undistinguishable than some normal sets.
