Motivation
There is presumably no single and widely accepted formal definition of structured sets = sets plus structure based on sets as primitive objects, but several approaches are around. See e.g. structures (model theory), echelons (Bourbaki), frames (Moschovakis). I don't want to discuss these approaches, but for simplicity's and specifity's sake I want to pick out one especially simple definition of rather generic structured sets — graphs (which among other things are able to interpret any structured set).
When we start with the „graph“ of sets $U = \langle V,\in\rangle$, sets as primitive objects are „defined“ just by
$$\text{Set}(X) :\equiv X\ \epsilon\ V$$
with $\epsilon$ indicating class membership. Graphs (structured sets), then, are - rather sophisticatedly - defined by
$$\text{Graph}(X) :\equiv (\exists S,R\ \epsilon\ V)\ R \subseteq S^2 \wedge X = \langle S,R\rangle$$
To complete the picture we define
$$\text{Relation}(X) :\equiv (\exists S\ \epsilon\ V)\ X \subseteq S^2$$
Thus, a graph is a set plus a relation over it, usually written as $G = \langle V,E\rangle$.
Interlude
Oppose this set-based picture to the category $\mathsf{Graph} = \langle \mathcal{O},\mathcal{M},\dots\rangle$ with $\mathcal{O}$ the class of all graphs as primitive objects, related by graph homomorphisms $\mathcal{M}$, etc. Thus graphs as primitive objects are defined by
$$\text{Graph}(X) :\equiv X\ \epsilon\ \mathcal{O}$$
To define „set“ as a now derived concept one might try to resolve the above „equation“ informally to obtain sets = structured sets minus structure. Thus, sets don't have any structure (anymore), so morphisms as structure-preserving functions don't have to preserve any structure (anymore), so every function from a set to any other graph (= structured set) is a morphism in $\mathcal{M}$. According to the standard definition of graph homomorphism, the graphs being sets are exactly the edgeless graphs (which complies with intuition).
Translating this into categorical terms we obtain:
$$\text{Set}(X) :\equiv (\forall\ Y\ \epsilon\ \mathcal{O})(\exists\ f\ \epsilon\ \mathcal{M})\ f: X \rightarrow Y$$
Making use of categorical terminology we can equivalently say: Let $\mathcal{C}/$ be the quotient category of $\mathcal{C}$ which identifies all morphisms (if present) from $A$ to $B$ as one. Then:
$$\text{Set}(X) :\equiv X\ \text{is an initial object in } \mathsf{Graph}/$$
Question(s)
I am aware that I didn't even mention the category $\mathsf{Set}$ of sets and the notion of a concrete category (even though $\mathsf{Graph}$ is a concrete category).
What I'd like to learn is - among other things - how the above definition of being a set relates to the usual notions:
Which concrete categories don't have sets?
Which non-concretizable categories do have sets?