This is indeed true, even more general: Let $i : Y \to X$, $j : Y' \to X'$ closed immersions of $S$-schemes and $F \in \mathrm{Qcoh}(Y)$, $G \in \mathrm{Qcoh}(Y')$. Then there is a canonical isomorphism $i_* F \boxtimes_{X,X'} j_* G \cong (i \times j)_* (F \boxtimes_{Y,Y'} G)$.

Proof: The commutative diagram

$$\matrix{Y \times_S Y' & \stackrel{i \times j}{\rightarrow} & X \times_S X' \\\\ \mathrm{pr}_1 \downarrow ~ & & ~ \downarrow \mathrm{pr}_1 \\\\ Y & \stackrel{i}{\rightarrow} & X}$$

gives a canonical homomorphism $\mathrm{pr}_1^* i_* F \to (i \times j)_* \mathrm{pr}_1^* F$ (which is not an isomorphism!), similarly for $G$. Thus we get a canonical homomorphism

$$i_* F \boxtimes_{X,X'} j_* G = \mathrm{pr}_1^* i_* F \otimes \mathrm{pr}_2^* j_* F \to (i \times j)_* \mathrm{pr}_1^* F \otimes (i \times j)_* \mathrm{pr}_2^* G$$

$$\to (i \times j)_* (\mathrm{pr}_1^* F \otimes \mathrm{pr}_2^* G) = (i \times j)_* (F \boxtimes_{Y,Y'} G).$$

This is an isomorphism: Since we have given it globally (and its formation commutes with (co)restriction of open subsets), we may assume that everything is affine, say $S=\mathrm{Spec}(k)$, $X=\mathrm{Spec}(A)$, $X'=\mathrm{Spec}(A')$, $Y=\mathrm{Spec}(A/I)$, $Y'=\mathrm{Spec}(A'/I')$ and $F = \tilde{M}$, $G = \tilde{M'}$. Then the homomorphism looks as follows:

$$(M \otimes_k A') \otimes_{A \otimes_k A'} (A \otimes_k M') \to (M \otimes_k A'/I') \otimes_{A/I \otimes_k A'/I'} (A/I \otimes_k M')$$

which maps $(m \otimes a') \otimes (a \otimes m') \mapsto (m \otimes [a']) \otimes ([a] \otimes m')$. In order to show that it is an isomorphism, it suffices to prove that $(m \otimes i') \otimes (a \otimes m')$ vanishes in the left tensor product for all $i' \in I'$ (similarly for $I$). But this is clear, since we can rewrite this tensor as $(1 \otimes i')(m \otimes 1) \otimes (a \otimes m') = (m \otimes 1) \otimes (a \otimes i'm')$ which vanishes because $M'$ is a $A'/I'$-module.

There should be a more global proof of this (valid in a topos context etc.), but I'm too lazy to write it down.