Nice question, Jonas!
Yes, in the case that $V$ satisfies ZFC and $M\subset V$, then indeed $M[j''\text{Ord}]=V$. To see this, consider first the case of a set of ordinals $A\subset\theta$ in $V$. Notice that from $j''\theta$ we may reconstruct $j\upharpoonright\theta$. Further, $j(A)$ is in $M$, and from $j(A)$ and $j\upharpoonright\theta$ we may easily reconstruct $A$ itself. So every set of ordinals in $V$ is in $M[j''\text{Ord}]$. If $V$ satisfies ZFC, then this suffices, since every set is coded by a set of ordinals. Namely, (as you know) if $a$ is any set, then $\langle \text{TC}(\{a\}),{\in}\rangle\cong\langle\theta,E\rangle$ for some cardinal $\theta$ and some binary relation $E$ on $\theta$, and then using a Gödel pairing function to code $E$ as a single set $A\subset\theta$. So for any set $a$, we find $E$ and then $A$, and by the previous argument $A$ is in $M[j''\text{Ord}]$, and so also $E$ and hence $a$ itself is there. Thus, $M[j''\text{Ord}]=V$, as desired.
The argument relies on the axiom of choice, and in the most general case, I believe this is required.