$\newcommand\Ord{\text{Ord}}\newcommand\Z{\mathbb{Z}}\newcommand\Q{\mathbb{Q}}$This is a nice question, and I don't have much to say about it,
except that I did want to mention that it is important in your
account that you are talking about standard models, which I take
to mean well-founded or transitive models. If you dropped that,
the phenomenon would disappear, for the following reason.

**Theorem.** For any consistent theory $T$ extending ZF, there are
two models of $T$ with the same ordinals, such that they are not
isomorphic by any isomorphism fixing the ordinals.

**Proof.** Let $M$ be any countable computably saturated model of $T$.
It follows that the natural numbers of the model $\omega^M$ are
nonstandard and have order type $\omega+\Z\cdot\Q$. The ordinals
of $M$ have type $\Ord^M$, which is the same as
$\omega^M\cdot\Ord^M$, by an internal isomorphism, and this has order type
$(\omega+\Z\cdot\Q)\cdot\Ord^M$. It follows that there is an
order-automorphism $\pi:\Ord^M\cong\Ord^M$ that shifts the ordinals within the nonstandard
parts of these blocks of $\omega^M$ by one (or one can make more
complicated automorphisms). That is, we shift all the $\Z$-chains by one. Since $\Ord^M\subset M$, we may extend
$\pi$ to an isomorphism $\pi:M\cong N$ to some model $N$, where $M$
and $N$ have exactly the same ordinals, but where many ordinals
$\alpha$ that are even in $M$ are odd in $N$ and vice versa (this
will be true exactly for the ordinals that are a nonstandard
natural number successor of the largest limit ordinal below them).
Thus, $M$ and $N$ have exactly the same ordinals, but are not
isomorphic by any isomorphism fixing those ordinals. **QED**

In particular, if ZFC is consistent, then there can be models of
ZFC+V=L that have exactly the same ordinals, but think different
things are true of those ordinals.