Here are a few observations about CORE.
Claim: It is consistent that CORE is not pairwise upwards directed.
Proof:
Let $\mathbb{P}_0$ be the class forcing for the Easton product of the Cohen forcing $\mathrm{Add}(\alpha^+, 1)$, over all cardinals $\alpha$, in $L$. Let $G$ be $L$-generic for $\mathbb{P}_0$. Then, we can pick a partition $A \cup B$ of the class of cardinals, with both $A, B$ being proper classes. Then $N_0 = L[G \restriction A], N_1 = L[G \restriction B]$ are both in CORE, and have no common upper bound in CORE.
On the other hand:
Claim: It is consistent that CORE is upwards directed:
Proof: Let $\mathbb{P}_1$ be the backwards Easton support iteration of $\mathrm{Add}(\alpha^+, 1)$ for $\alpha$ successor of a singular cardinal in $L$. Let $G$ be an $L$-generic and let $M=L[G]$.
Sub Claim: For every $N \in CORE^M$, there is an ordinal $\alpha$ such that $N \subseteq L[G \restriction \alpha]$. In particular, CORE is upwards directed.
Sketch of Proof: Let $\alpha$ be minimal (successor of singular) such that $L[G\restriction \alpha] \not\subseteq N$ and let $x\in N$ be a set of ordinals of minimal rank such that $x \notin L[G \restriction \alpha]$. Then $x$ has to be a fresh set over $L[G \restriction \alpha]$, and by minimality if has to be a subset of $\alpha^+$. By gap-type arguments, $x$ together with its name (which is in $L$) codes the set $G \restriction \alpha$, and thus $L[G \restriction \alpha] \subseteq N$.
Finally, large cardinals seems to have negative affect on the directedness of CORE:
Claim: Let $\kappa \in M$ measurable and $2^\kappa = \kappa^{+}$. Then, there are $N_0, N_1 \in CORE^M$, and $x \in N_1$, $y \in N_0$, such that $N_0[x] = N_1[y] = M$.
Proof: Let $\mathcal{U}$ be a normal ultrafilter on $\kappa$ and let $N$ the ultrapower by $\mathcal{U}$. Let us construct inside $M$ two $N$-generic filters $G_0, G_1 \subseteq \mathrm{Add}(\kappa^{+}, 1)$, such that their bitwise xor codes $\mathcal{U}$. This is possible, since $2^\kappa = \kappa^{+}$ (both in $M$ and in $N$). Let $N_0 = N[G_0], N_1 = N[G_1]$ and note that $N_0[G_1] = N_1[G_0] \supseteq N[\mathcal{U}] = M$. QED
On the other hand, if $A$ is a set of ordinals and $A^{\#}$ exists, then $L[A] \in CORE$, since no set forcing in $L[A]$ can introduce a sharp for $A$.
So under the large cardinal axiom "every set has a sharp" (which follows from the existence of class of measurable cardinals, for example), $\bigcup CORE = V$, so CORE can certainly contain sets which are not in the mantle.